Robomongo has been acquired by 3T

MongoChef is now Studio 3T. Click here to read the full blog post about “Robomongo has been acquired by 3T” on studio3t.com!

How to export your MongoDB data to SQL with Studio 3T Pro or Studio 3T Enterprise

What is new in Studio 3T 5.0

Studio 3T is the new name of MongoChef and we are more excited than ever to announce the major release of Studio 3T 5.0.

In this release we focused on improving the experience for those that are discovering Studio 3T and downloading it for the first time. Of course, we also added plenty of improvements and new features to the application as well. For example, we introduced an important fix for the SSH connectivity (that will vastly improve the connection set-up speed for many users). You can now also import CSV directly from the clipboard and export selected documents via mongodump. With these updates, Studio 3T is not only getting better but also expanding the visibility over its rich set of tools and features.

A few words about the name change

Studio 3T is becoming more than a simple GUI. It is now a mature product which has become the most popular commercial IDE for MongoDB. We needed a bigger, roomier name to do justice to the growing capacity of the studio with each new release and this new name is Studio 3T. Visit this page to know more.

What changes in the download experience?

Everything is now in one app. Starting with Studio 3T 5.0, you no longer need to worry about making sure that you download the correct file corresponding to the right app edition (i.e. Core, Pro, Enterprise). While we used to have dedicated apps for each edition, we have now unified all editions under one download package. It is now the license key within the app that decides which tools and features are enabled and which ones require an upgrade.

We believe this is going to make things much easier for those discovering Studio 3T for the first time. Also, it is going to be much easier to move between various editions of Studio 3T as your needs grow over time. Please contact us if you are interested in our upgrade program. We will be happy to assist you in transitioning to one of our premium-feature software packages.

3T MongoChef is now Studio 3T

Renaming popular and successful products is a fraught business. Everyone’s heard of famous fails like the ‘New Coke’ debacle, or the switch from the much-loved ‘Royal Mail’ in the UK to the breathtakingly bland ‘Consignia’ – a rebrand that became a re-re-brand when they quickly reverted to ‘Royal Mail’. Even Australia’s proudest savoury spread – Vegemite – had its own brush with disaster when in 2009 they tried to rebrand under the satirically suggested name “iSnack 2.0”.

Second time around

At 3T we’ve been developing tools for MongoDB professionals for three years now. That might not seem very long but MongoDB has only been around since 2007, and boy has it grown. Its popularity continues to grow at an astonishing pace. Our first edition of the MongoChef GUI was originally called 3T Data Manager and in 2014 we changed the name to MongoChef. Since then, more than 100,000 users have got on board and got more work done, faster and more securely, using our graphical workspace. We’re constantly updating it, with new features every fortnight over the last twelve months, and our original GUI has grown up substantially. It’s on a roadmap now to becoming a full-fledged IDE (integrated development environment), with a steadily increasing set of features and functions, including the data comparison and synchronisation tools that we originally offered as separate tools. In 2015 we added security authentication with LDAP and Kerberos to our Enterprise edition. See here for a full feature list.

Security a must-have

If you’re using MongoDB for rapid prototyping of an application, or in non-production, non-commercial staging and test environments, then security might not be a huge issue. But MongoDB is now a mature and dependable platform. It’s deployed in production environments by organisations small, medium and humongous, across all industries. Security is no longer a ‘nice-to-have’, it’s an acknowledged ‘must-have’. Implementing all the on-board security features and even just setting user authentications is a whole lot easier and more intuitive to do in an easy-to-use graphical environment.

Management studio or IDE

What started out as a quick and easy way to work more enjoyably than with the raw MongoDB shell, has over three years become a much more sophisticated beast. We’ve decided the time is right to adopt a name that gets closer to the experience of working inside this IDE. More and more of our customers are running whole teams that work predominantly with MongoDB and more than a few of them have made the comparison with Microsoft’s SQL Server Management Studio (SSMS) that they’re well accustomed to working inside. Studio 3T is ideal for teams needing to standardise their working practices to improve consistency and accuracy.

Immensely satisfying

Whether Studio 3T is best described as an IDE or a management studio, a client, a GUI or a workspace, is a subject probably of interest only to the self-declared pedants and fusspots inside 3T. But the fact that there are over 100,000 MongoDB professionals using Studio 3T on a regular basis tells us that it’s doing a job that really does need doing: it makes dull and fiddly jobs easy and quick to tick off your list; it makes complex jobs like building aggregation queries a joy to accomplish; it makes securely protecting your MongoDB estate immensely satisfying. Studio 3T frees up a load more hours in the day for developers and data professionals, data analysts and BI specialists alike, so they can get on with doing the stuff that they should be focusing their skills on.

Studio 3T is always being improved. It’s graduated from being a GUI. We think it’s well on its way to becoming an IDE, so that’s what we’re calling it from now on. Studio 3T for MongoDB is the most widely used NoSQL IDE in the world. It’s a place where we can all get more done.

New Aggregation Pipeline Operators

MongoDB 3.4 has brought out a number of new, very useful aggregation operators for both recursive search ($graphLookup), and faceted search ($bucket, $bucketAuto, $facet, $sortByCount).

See below for a brief explanation of each new operator. And of course, both IntelliShell and the MongoChef aggregation builder fully support the new additions.

OperatorDescription
$bucketCategorizes incoming documents into groups, called buckets, based on a specified expression and bucket boundaries.
$bucketAutoCategorizes incoming documents into a specific number of groups, called buckets, based on a specified expression. Bucket boundaries are automatically determined in an attempt to evenly distribute the documents into the specified number of buckets.
$facetProcesses multiple aggregation pipelines within a single stage on the same set of input documents. Enables the creation of multi-faceted aggregations capable of characterizing data across multiple dimensions, or facets, in a single stage.
$sortByCountCategorizes or groups incoming documents by a specified expression to compute the count for each group. Output documents are sorted in descending order by the count.
$graphLookupPerforms a recursive search on a collection. To each output document, adds a new array field that contains the traversal results of the recursive search for that document.

MongoChef not only fully supports the MongoDB aggregation pipeline, it supports a whole range of functions designed to decrease the time it takes for you to execute your tasks. If you would like to experience these time-saving features then start your 14-day free trial. And if you’re on your smartphone, or don’t have time to download now, simply fill in the form below and we will email you a link for later.

And of course, MongoChef Core is forever free for non-commercial use!


No time to try it now? Enter your email to get a link for later.

 

How to rename a field in MongoDB using MongoChef

Every now and then you may need to change the name of a field within your MongoDB documents. Whether you need to do it for a single document, or perhaps for multiple documents, this task can be easily accomplished with only a few steps from within the MongoChef app.

Renaming A Field Name In A Single Document Within MongoDB

The simplest of these tasks is changing a field name in single document. See the steps below:

  1. Open the document in which the field to be renamed is located
  2. Double click on the field name (In this case “dob”)
  3. You will now have the option to type the new field name into this space
  4. Once you press “Enter” the new field name will be saved

MongoChef Support, MongoChef, MongoDB

Renaming A Field Name Within Documents Matching A Query

Should you wish to change all documents matching a particular query then this can also be easily achieved in just a few steps. In this example we are changing the “dob” field in all documents relating to customers with the last name “Miller”. See below:

  1. Type in your query: {last:"Miller"} – and hit “Enter”
  2. Open one of the documents resulting from the query search
  3. Find the field that you would like to rename (“dob”) and right click it.
  4. Select “Rename Field”
  5. Type in the new field name (“date_of_birth”) and hit“Enter”
  6. Select “Documents matching query criteria” from the dropdown box
  7. Click “Enter”
  8. Now every document within this query will have the field name “date_of_birth” in place of “dob”

MongoChef Support, MongoChef, MongoDB

Renaming A Field Name In All Documents Within A Collection

Finally, to rename a field in all documents within a collection follow the steps below (identical to the steps above apart from no. 4):

  1. Find the field that you would like to rename (again “dob”) and right click it
  2. Select “Rename Field”
  3. Type in the new field name (“date_of_birth”) and hit “Enter”
  4. Select “All documents in collection” from the dropdown box
  5. Click “Rename”
  6. Now every document in the entire collection will have the field name “date_of_birth” in place of “dob”

MongoChef Support, MongoChef, MongoDB

 

That’s it for this post, but if you still have a little time, please check out the other very useful features of MongoChef!

Real-Time Server Status Charts for MongoDB in MongoChef

This blog post demonstrates the Server Status Charts feature of MongoChef for MongoDB.

The server status charts show real-time updates of what’s happening on a MongoDB instance:
server-status-charts-allso you can easily monitor particular things of interest on production, development, test, or local instances.

You can open a Server Status Charts tab by right-clicking on a connection (or database, or collection) and select ‘Server Status Charts’ from the ‘Show Server Info’ menu.
server-status-charts-context-menu
You can customise which charts are shown, by clicking on ‘Select Charts’ in the upper right hand side corner of the Server Status Charts tab:
select-server-status-chartsFor example, if we wanted to focus on Operation Counts, Current Connections and Current Network Traffic, we would select only those, and the charts will be laid out intelligently to accommodate them:
selected-charts

What the chart information means

In this section we’ll explain the information shown by each of the different charts.

Note that the information displayed in the charts may be:

  • Cumulative – where the charts show information since the start of a particular point until the present moment, e.g. all events of a particular type since the database started.
  • Per Update Period – where the charts show information about a particular event that occurred during an update period, e.g. if we have the ‘Update frequency’ set to ‘2 sec’, then the chart will show all events of that particular type that occurred within 2 second periods.
  • Point in Time – where the charts show information about a particular event as it was at the point in time that the server status was requested from the database.

The table below details each of the server status charts:

Operation Counts
Per Update Period
The number of database operations by type during the update period.
Active Clients
Point in Time
The number of the active client connections performing read and write operations.
Current Queue
Point in Time
The number of operations that are currently queued and waiting for the read or write lock.
Current Connections
Point in Time / Per Update Period
The number of incoming connections from clients to the database server. ‘In use’ and ‘Available’ report the point in time values, ‘Total created’ shows values per update period.
Current Network Requests
Per Update Period
The number of distinct requests that the server has received during the update period.
Current Network Traffic
Per Update Period
The number of bytes that reflects the amount of network traffic received by and sent from this database during the update period.
Total Connections Created
Cumulative
Count of all incoming connections created to the server. This number includes connections that have since closed.
Total Network Requests
Cumulative
The total number of distinct requests that the server has received.
Total Network Traffic
Cumulative
The total number of bytes that reflects the amount of network traffic received by and sent from this database.

More detailed information about the counters the chart data are based upon can be found in the MongoDB documentation.

That’s it for this post, but if you still have a little time, please check out the other very useful features of MongoChef!

MongoDB Schema Discovery and Exploration with MongoChef

One of the great things that we love about MongoDB is of course that it is schema-less. This makes adapting your application to changing requirements a breeze. That said, your data will often have a fixed implicit schema – e.g. each document in your employees collection will likely always have a first and last name field. So, making sure from time to time that all documents indeed contain certain fields is probably a good idea. Likewise, having the power to dynamically add new or discontinue old fields in your documents can often lead to a proliferation of various “versions” of your document schema. So, getting a feel for how often a certain field occurs in your collection can be quite helpful.

Luckily, MongoChef Pro makes discovering and exploring the schema in your MongoDB collections super-easy! With MongoChef, you can quickly:

  • check the health of your schema
  • find schema anomalies
  • inspect data outliers
  • visualize data distributions

So, let’s dive right in!

Schema Discovery

Select the collection whose schema you want to explore (in this example “customers”) and click the “Schema” icon in the global toolbar. This will open a tab where you can now configure your schema discovery:

mongodb schema discovery

  1. Here you can control how MongoChef should sample documents from the collection for the schema discovery. By default MongoChef will analyze randomly selected documents. You can choose between “Random”, “First”, “Last”, or “All” – in which case MongoChef will read in all documents in the collection.
  2. MongoChef gives you full control over how many documents should be read for the schema discovery. For our example, we will look at 1,000 documents.
  3. By default, MongoChef will not analyse the elements of any array fields when it encounters them. The reason is that arrays can often contain thousands of elements of the same type which can lead to bloated schema results. You can of course override this default behavior.
  4. You can also provide a query to further control the document set that should be used for the schema discovery. In our example, we will use an empty query, which will return all documents in the collections
  5. Click “Run analysis” to start.

After the analysis has completed, you will see the schema discovery result page:

mongodb schema discovery result

In the left-hand pane (1), you will see the discovered schema tree. For each field you see its name, its global probability – i.e. which percentage of documents were found to contain that field, and its discovered field type(s). You can now easily explore your schema as you would with a JSON document.

In the right-hand pane (2), MongoChef displays information about the type and data distribution of the currently selected schema field.

Verifying Your Schema

The schema tree is a great tool to understand and verify your schema. For our example, we assume a – fictitious – “customers” collection which contains the personal information of the customers of our – equally fictitious – shop. With the schema tree, we can now easily verify that required fields like “name” and “transactions” do indeed occur in 100% of our documents. We can also observe that the optional field “title” is apparently provided by 56.7% of our customers. The schema tree is also really helpful in discovering schema anomalies:

Discovering Missing Fields

mongodb schema discovery missing fields

Looking at our schema discovery results, we see that field “first” – which stores our customers’ first names – is missing in 0.4% of our documents. This may for example suggest that our web shop software might be flawed. To learn more about those documents that are missing a “first” field, MongoChef makes it super-easy to explore that actual data. If we right-click the “first” field, we can choose “Explore documents not containing selected field”. This will open a new query tab that shows all documents that do not contain the field “first”. It is important to note that this query will of course honor the base query criteria of your analysis but will bring up all matching documents in the collection, not just in the (limited) sample set.

mongodb schema discovery inspect missing fields

We could now inspect those documents which might reveal clues as to what has caused the missing field anomaly.

Discovering Unexpected Fields

Looking at our schema discovery results again, we spot another anomaly – an unexpected field:

mongodb schema discovery unexpected field

We see that in 95% of our documents, the field “user_name” is spelled with an underscore. Yet, in 5% of documents, it is spelled with a hyphen. This could for example be down to a typo somewhere in the source code. After fixing the typo in the code, MongoChef then makes it very easy to also fix it in your collection. Right-click the incorrectly-spelled “user-name” field and select “Explore documents containing selected field”.

mongodb-schema-discovery-explore-documents

This will open a new query tab showing all documents containing the selected field (“user-name”). Here right-click the field to rename it:

mongodb schema discovery rename field

and choose “All documents in collection” in the ensuing dialog to rename all occurrences in the collection:

mongodb rename all

Easy-peasy :-)

Discovering Incorrect Field Types

Another type of schema outlier that one commonly wants to look out for is incorrect field types. MongoChef makes that really easy to spot too. Consider in our example the “address” field in our customer documents. We store the addresses as an embedded object of the following type:

“address”:{“street”:{“name”:”Scofield”,”suffix”:”Drive”,”number”:”34″}}

However, when we look at the “address.street” field in the discovered schema tree, we can see that there appear to be some outliers where “address.street” is of type String:

mongodb schema discovery outlier

When we select the String instance of “address.street” in the schema tree, we can quickly see in the right-hand data pane that in 4 documents something must have gone awry and all street information was stored in a simple string.

mongodb schema discovery type outlier

We can right-click the String instance node in the schema tree to “Explore documents with selected field of type String” to have a closer look at those documents:

mongodb schema discovery explore documents

 

mongodb schema discovery document view

Exploring Data Distributions

As we have seen, when you click a field or one of its type instances in the schema tree, you can see charts showing – depending on the selected data type – various statistics on the type or data distribution of the field. Now, while MongoChef is not a full-blown BI tool by any means, these data distribution charts can often already give you some very useful insights into your documents.

Value Histograms

For numeric fields, one of the stats charts that the right-hand pane shows is the value histogram. If we look for example at the value histogram of our “transactions” field, we can quickly observe that most customers have engaged in around 50 transactions, with tails of less common numbers of transactions.

mongodb schema discovery histogram

Top Values

For many data types, the right-hand panel also shows the top values that were found across the analyzed documents. This can often be helpful to spot data outliers. Consider the “package” field in our example. Suppose our customers can subscribe to a “Free”, “Basic”, “Standard”, “XL”, or “XXL” package. However, we see that there are some customers who seem to have a “Beginner” package – which may indicate a backend glitch, for example.

mongodb schema discovery top values

Luckily, we can now use MongoChef to edit those values directly in-place:

mongodb schema discovery in place editing

Date Distributions

For date fields, MongoChef shows you in detail the value distributions. Consider, in our example, the “registered_on” field. If we look at its “Monthly value distribution”, we notice that customer registration seems to be particularly strong in the summer as well as in January. This might then provide valuable feedback to the marketing and sales teams.

mongodb schema discovery date distribution

Summary

MongoChef provides a very powerful schema explorer feature that lets you easily discover the schema that is present across the documents in your collection and thereby helps you find schema and data outliers. Drilling down into individual fields, you can see, for each field, various visual statistics relevant to the data type of that field.

MongoDB schema discovery and exploration has never been easier!