To provide a uniform high-performance lab experience for all users, the NMR Facility will soon be migrating all spectrometers’ software and operating systems to the same platforms: Topspin 3.6.2 and Centos 7. When complete, each user’s Topspin interface should be the same on every manually-operated spectrometer, enabling them to make decisions about which instrument to use based on capability and performance, not habit or software limitations.

This is a significant change in our computing environment, so we will take advantage of the opportunity and pare down old, unused user accounts and data folders. While it is always important for each user to back up their data, it is now especially important for all labs to back up the data of group members who left before January 1, 2015, so their data is secured in the lab.

Current State:

Each of our five spectrometers is running a different combination of operating system and Topspin.

• 400-1, installed in 2016, is on CentOS 7 and, after an upgrade during the summer of 2019, is running Topspin 3.6.2, the latest available for Avance consoles. This is our target state, though the manually-operated instruments will not be running Icon-NMR by default.
• 400-2, unfortunately, cannot be upgraded from CentOS 4 and Topspin 1 – the console will not support anything newer; this instrument is scheduled to be replace mid-2020 anyway, and it will be shifted to automation.
• All 500’s are running on CentOS 5, but that is no longer supported and a few years ago a security hole was discovered in it; with no further patches available for it, the only fix is to upgrade.
• 500-3 is using Topspin 2.1, which lacks a number of the innovations and experiments that come with Topspin 3. Consultation with Bruker engineers has determined that 500-3’s Avance II+ console is the oldest model supportable by Topspin 3.6.2. Whew! Looking forward to some modernization for 500-3’s users!
• 500-2 is already using Topspin 3.6.2. In the early fall, we had some trouble implementing the edited, sensitiviy-enhanced HSQC on this instrument, and we upgraded the software in part to take care of that.
• 500-1 is running Topspin 3.1.

Barriers to Migration

Upgrading a CentOS computer to a new operating system is not nearly as straightforward as it is with a PC or Mac. After consultation with PSD Computing and other experts, it’s clear we need to start with a new hard drive on each system, install a disk image, kindly provided by Bruker, that has the correctly-configured version of CentOS 7 and Topspin 3.6.2 on it, re-create all the user accounts and copy their data from the previous hard drive. It’s pretty complicated and labor intensive, but we’ve been piloting the process in the Searle support lab to work out the details.

Legacy Accounts, Data

Note in the above paragraph that all user accounts have to be re-created from scratch (by scripting), and data copied over for continuity. It’s a big job, and it would be best if we limit the accounts we re-create to those that are actually useful – those corresponding to current users and those who only left UChicago relatively recently.

Currently, our spectrometer computers include hundreds of legacy accounts and data files that have been copied from one system to another over the years, some dating back as far as 2003. Hundreds of data folders are not even owned by user accounts, and have not been used since the mid-2000s.

It is time to delete data files with no owners, and user accounts that have not been used since January 1, 2015.

All data have been backed up several ways over the years, and all spectrometers are backed up to an admin-access backup computer every night. Deleting these files will not make the data completely unavailable.

Back up your lab’s data

That said, every lab should retain copies of data from all group members, including those who left years ago. Data on the spectrometers has permissions set so that members of the same group may read, and therefore copy, data from other lab members.

If your lab has not done so in a while, it would be best if a lab representative relatively comfortable with the Linux command line were to copy all the lab’s data to a USB drive, then give that drive to the lab’s PI.

How do you find all the data belonging to a lab?

• In a Terminal window, log in to the spectrometer and navigate to /opt/topspin/data.
• Type:
  ls -l | grep <labname>
  (for example, ls -l | grep snyder ) Note that the vertical line is a “pipe”, found toward the upper-right area of your keyboard.
• This will list all the data folders owned by people in that group. If you like, you can highlight that text and copy/paste it to a text file so you have a record of all the user accounts you’re working with.
• Do this for every spectrometer. While most users have accounts on all instruments, a significant minority do not.

How do you copy others’ data to nmr15 or nmr8?

• The permissions on the spectrometers are set so people from the same group may ready and copy each others’ files, but not overwrite or delete them.
• In a Terminal window, log in to a data transfer computer, either nmr15 (via nmr50 from outside the lab) or nmr8 (nmr60 from outside the lab).
• It is strongly suggested that you create a new directory/folder for collected lab data.
  • To create one called “labdata”, type: mkdir labdata
• Navigate into that directory
  • Type: cd labdata
  • Type: pwd
    • This tells you what directory you’re in, which should be /home/<your_username>/labdata
• For each user in your group, for each spectrometer, copy the user’s data folder to your new directory on the data transfer computer. Let’s assume you’re logged in to nmr15 in Searle.
  • Type: rsync -crv nmr500-1:/opt/topspin/data/<username> .
  • Substitute the username of interest for <username>.
  • Don’t forget the “.” at the end. In linux, that means “right here”. The rsync command thus copies the source directory (specified by /opt/topspin/data/username) on the remote computer (specified by its computer name, here nmr500-1) to the destination “right here” (specified by “.”, which should be /home/<your_username>/labdata).
  • You should see a lot of lines appear in your terminal window indicating files being copied.
  • Type ls to make sure that user’s directory appears.
  • Type ls <username> to check the contents of that user’s directory, It should at least have an “nmr” directory inside. 400-1 data normally appears here alongside the “nmr” directory.
  • Type ls <username>/nmr to check the contents of the nmr directory. Sample folders normally appear here, except those taken on 400-1.
  • REPEAT for another spectrometer in the room: rsync -crv nmr500-2:/opt/topspin/data/<username> .
  • Check the contents of the user’s folder to see whether the new data appeared (you’ll immediately see whether the number of sample folders grew.
    • Type: ls <username> , then type  ls <username>/nmr
  • REPEAT for 400-1.
• REPEAT for all users in the lab.
• REPEAT the process for GCIS instruments, logging in to nmr8 (via nmr60).

How do you copy others’ data from nmr15 or nmr8 to a USB drive?

The normal process here is to use a graphical utility like Filezilla or Cyberduck.

Log in to either nmr50 or nmr60, locate you backup folder, labdata, and drag it over to the USB drive. It will probably take a while to copy, possibly an hour or two. Repeat with the other data transfer computer, nmr60 or nmr50.

Next Steps

Once data are backed up, certain directories and accounts on spectrometers will be deleted.

1. First to be deleted: data directories last updated before 2010 AND belonging to groups no longer at UChicago
2. Next: data directories last updated before January 1, 2015 AND belonging to groups no longer at UChicago
3. Next: data directories last updated before 2010 AND with no account owners
4. Next: data directories last updated before January 1, 2015 AND with no account owners
5. Next: all remaining accounts with data directories last updated before 2010
6. Next: all remaining accounts with data directories last updated before 2011, then 2012, then 2013, then 2014.

Linux ID reconciliation

Once the old accounts are deleted, the remaining accounts need some administrative harmonization. In Linux, every user and group gets a userid number and groupid number, respectively. For the facility to function smoothly as an integrated system, we need each person’s user account to have the same userid on all systems, and all groups to have the same groupid on all the systems. This is mostly true in the facility today, but some IDs are different on different computers. This has caused some trouble when doing billing and needs to be fixed. Users and groups needing harmonization have already been identified, and changes will be made to the users’ accounts and data folders that should be invisible to the user.

New system creation

Once all the user and groups accounts are harmonized, new hard drives will be prepared offline in the support lab for 500-1, 500-2, and 500-3, using the disk image provided by Bruker. This will involve transferring instrument-specific information to the new system. User accounts will be created by script for each, and existing data will be transferred in a process that will occupy a few hours of instrument time.

Testing each new system should not take much time. We’ll know within a half hour whether the new system is running the spectrometer correctly. If it isn’t right, we’ll just switch the old hard drive back in, as if nothing had happened.

Finished!

When the migration is all done, users will see the same environment on all manually-operated instruments except 400-2. 500-3 will get new capabilities, including much better VT experiment management.

Much of the improvement will be invisible to the user, but it will make facility operations more robust. This will represent something of a “reset” from which the facility can grow into the future.