Archive for November, 2016

MG-RAST version 4.0 released today

Tuesday, November 15th, 2016

Version 4.0 is here!

After several months of beta testing and significant user feedback, we have today released the new version at

The old version is still available at (note no new data will be available in the old version).

Follow us on Twitter @mg_rast

Highlights of the new version include:

myData – access all your data in one place

The new myData page now summarizes all information about your data. It shows you your running jobs, completed studies, news and more. It is an entry point all main MG-RAST functionality. You can use the toggle bar at the top right to show and hide the different sections. Each section has links to a more detailed view or editor of the according data items. You can also use the search bar at the top to quickly find the datasets you are looking for.

analysis revamped

The new analysis page was built to cope with the growing size of datasets. Instead of waiting for the results of each parameter change, the complete abundance profiles of your selected data sets will be downloaded to your machine. You can then filter, drill-down, slice-and-dice your data all you want in an instant. Many different interactive visualizations are at your fingertips to explore your data. The plugin section offers additional tools to view your data.

Note that the RAM on your computer limits the number of data sets you can include in your analysis.

The profiles you downloaded can be saved to your harddrive and reloaded in a later session. This gives you instant access without the need to download from our server again.

MetaZen 2.0 — Improved metadata support with better feedback

Getting all set with the metadata for your project has just become a lot easier. The new version of metazen will give you direct feedback and input help when creating metadata spreadsheets. You can download the results in Excel format or just send them directly to your inbox.

metagenome overview and processing receipts

The metagenome overview page offers the data you are familiar with in a fresh look. Every bit of data now has provenance information, as well as multiple download options including the images as well as the data used to produce them. The technical details of the data progression through the MG-RAST pipeline is linked on the processing receipt page. It shows general information as well as details about every pipeline step.

MG-RAST newsletter November 2016

Wednesday, November 9th, 2016


  1. Switching to new MG-RAST web interface  on November 15th 2016
  2. Update on processing backlog
  3. Feedback
  4. Miscellaneous


1. Switching to new MG-RAST web interface on November 15th 2016

After an open beta testing period and feedback from a significant number of users, we are now ready to transition away from the old MG-RAST web interface to a new more modern and more capable user interface.

–> will direct you towards the new style interface after November 15th. <– 

As many of you have tested the new interface already (at we feel confident that the new version will add significantly to the usability of the system, the old style interface will remain available for a while at (  The old style interface will not be showing any new data, but remain functional for a while.

2. Update on processing backlog

Due to both the massive demand for MG-RAST analysis and several unplanned power failures, we experienced problems with loading the results of the analysis pipeline (aka profiles) into the database that hosts said profiles. We have now transitioned the pipeline (as of November 9th 2016) to load into Cassandra instead of Postgres and are able to load data at a rate of 1 Gigabasepair / 60 seconds. We already have uploaded 30% of all public data to Cassandra and are continuing to transition all profiles to Cassandra in the coming weeks.


Technical details below:

We are working with a 3 step plan that we present here in the interest of transparency:

– step 1:

November 9th: Switch over API and pipeline to the new (v4 or “Cassandra” version) of the code base. Any new jobs created will automatically use the new code. The old style API will remain available for a while and we will provide automatic forwarding in the API. If the automatic forwarding does not work for you, you can reach the old style (“postgres” or v3) API at You should not need to use the temporary address, but some codes might not contain support for http redirecting.

The following API resources are being forwarded to the v3 API:



Consequently they will not be available for any new data sets being added to the Cassandra database until we have retargeted them. This work is going on now.

– step 2:

We are currently loading all public data sets into the v4 database and this process will continue for a while (15,000 data sets took 1 week to load).

A large number of data sets are currently being processed. All existing jobs (they are mostly waiting for the database load steps) are being rewritten to load into the v4 databases as well. This process will start November 10th 2016 and continue for a number of days.

– step 3:

On November 15th we will switch the CNAME to point to (The old style interface will remain available at

On or around November 15th we will deploy the code to retarget the /annotation API resource to the v4 database (Cassandra).

The /matrix API resource will be retargeted at a later date. NOTE: Until it is retargeted the /matrix API resource will only support data loaded prior to October 1st 2016.

3. Feedback

We will keep the survey for the new web interface  (located at ) open at Please continue to provide us feedback for the changed web interface. The more specific you are, the better we are able to build what you need.

4. Miscellaneous

  • We have updated the manual for version 4.0
  • We will remove all old content from the Wiki with the release of version 4.0