Remote observing with ULTRASPEC

Vik Dhillon, version: 20 December 2020


  1. Introduction
  2. Remote observing checklist
  3. Remote observing from Sheffield
  4. Remote observing from Warwick
  5. Remote observing from Chiang Mai
  6. Troubleshooting
  7. Contacts


Introduction

This manual describes how to observe remotely with ULTRASPEC at the TNT. It is possible to do this from three different locations - the NARIT headquarters in Chiang Mai, Sheffield and Warwick. This document is an attempt to bring the following information into a single place:

This document does not give you checklists on what to do in the afternoon/start-of-night/end-of-night, nor does it tell you how to take bias frames, focus the telescope, take flat fields, or what to do when something goes wrong with the instrument. Please refer to the ULTRASPEC user manual for such information. Lots of useful links, e.g. to the blog, logs, filters, pipeline manual, phase II, etc, are given on the left-hand side of the ULTRASPEC homepage. A few additional links that may be useful whilst remote observing:


Remote observing checklist

The section provides a checklist of things to remember when remote observing with ULTRASPEC on the TNT, whether that is from Sheffield, Warwick or Chiang Mai. For general checklists on what to do in the afternoon, start of night and end of night, please refer to the ULTRASPEC user manual.

The checklist below starts from a point where you have successfully followed your site-specific instructions (for Sheffield, Warwick or Chiang Mai) and you now see the DRPC desktop on your own local desktop:

Before starting

Start of night

During the night

End of night


Remote observing from Sheffield

This section contains specific information for remote observing from Sheffield.

  1. Prior to remote observing from the Department, please make sure you have the following: up-to-date fire, out-of-hours and Covid-19 training (if applicable); out-of-hours U-card swipe permission for the Hicks Building, E corridor and office E13B. Please also make sure that you have read and signed the Physics Health and Safety Policy document distributed via email by Richard Davis. Please also make sure that you have read the Remote observing with ULTRASPEC risk assessment, which was distributed by email. Finally, make sure that you have a mobile phone on you at all times, even when going to the toilet or tea room.
  2. The Sheffield control room is in the Astrophysics postdocs' room, E13B, on the first floor of the Hicks building. You will need your UCard to access the office out of hours - make sure you have out of hours access to the building! Also, make sure you sign in if you're working out of hours, including when staying late on a week day.
  3. If observing during a pandemic, use the cleaning materials provided in E13B to clean your hands and wipe down the work area prior to use, and do the same again when you're finished (including the office entrance door handles). Since you will almost certainly be observing on your own during a pandemic, please follow the lone-worker procedure detailed in the risk assessment. In particular, ensure you email vikcam when you enter the building to start observing, quoting your mobile phone number and the E13B phone number (0114 222 4541), and email again when you've finished observing and are about to leave the building. As a fail-safe, you should also set up a pre-scheduled email, available in gmail using the Schedule send option. This email should be timed to be sent to vikcam around sunrise (Thai time) and informs the team that someone should try phoning you (and, if necessary, University security) to check that all is ok. You must remember to cancel this scheduled email before you leave the building to prevent any false alarms. Of course, if you live with someone, you could instead tell them where you will be, when you are expected home, and ask them to phone University security on 01142224444 if you don't come home at the expected time.
  4. On the first desk on the left-hand side of the office sits europa, which you should log into using your astrogroup username and password.
  5. Once logged in, open an xterm and type:

    ssh -L 5900:localhost:5900 localhost -p 40002 -l observer

    You will be prompted for the password of the observer account on the DRPC - ask a member of the ULTRACAM team if you don't know this. The above command creates a tunnel to the ULTRASPEC DRPC in the control room at the telescope. Leave this window running.
  6. Next, open another xterm and type:

    vncviewer -geometry 800x600 localhost

    You will be prompted for the vnc password - ask a member of the ULTRACAM team if you don't know this. The above command opens a window on europa. If someone has already logged into the observer account on the DRPC in the telescope control room, you will be presented with the DRPC desktop, otherwise you will see the login screen of the DRPC (in which case you should log into the observer account as usual). It is recommended that you resize this vnc window so that it fills both monitors connected to europa.
  7. If all has worked, you are now essentially at the same place as if in the telescope control room in Thailand, albeit on a slow network. Type slowly, wait, and don't get ahead of yourself!


Remote observing from Warwick

This section contains specific information for remote observing from Warwick.

  1. Remote observing is carried out from a PC set up on the left-hand side of the "Cave" as you enter it.
  2. Log into the PC using the username ultraspec. For the password, ask Tom Marsh.
  3. Once logged in, open a terminal and type: tunnel. (See Troubleshooting if you are not.) You will be prompted for a password - ask Tom Marsh if you don't know it. This command sets up an ssh tunnel to the DRPC on port 50002. Once set up you can, for instance, ssh to the DRPC using a command like: ssh -p 50002 observer@localhost. Similarly, scp -P 50002 observer@localhost: . would copy the file of interest on the DRPC back to Warwick.
  4. Open another terminal (typically by doing this as a tab of the first). Type vnc (you may also need to add the "-geometry 800x600" flag) You will prompted for a password - if you don't know it, ask Tom Marsh. If successful, the DRPC login screen or desktop (if someone has already logged into the observer account in the telescope control room) should appear. Stretch this out to cover both screens (you might have to "unmaximise" since it only seems to maximise to one screen). If typing vnc fails and does not prompt you for a password, check that the ssh tunnel started up successfully.
  5. If all has worked, you are now essentially at the same place as if in the telescope control room in Thailand, albeit on a slow network. Type slowly, wait, and don't get ahead of yourself!


Remote observing from Chiang Mai


Troubleshooting

For problems specific to remote observing, please check the list below. For more general instrument problems, please refer to the troubleshooting section of the ULTRASPEC user manual.

ssh tunnel does not work

The DRPC may not be booted or the external network could be down. You need to ask the TO to check the latter by logging in as the observer user and using firefox to browse to a site at Sheffield (or Warwick, if observing from there), e.g. http://www.vikdhillon.staff.shef.ac.uk. If this page opens, then the link from Sheffield/Warwick should be fine. Note that the tunnel software is constantly monitoring the connection and if it drops it will automatically try again forever without any user intervention - rebooting the DRPC just delays this happening. Hence only reboot the DRPC as a last report if nothing else works.

vnc doesn't work

You may find that someone else is vnc'ed into the DRPC at the same time as you, e.g. from the control room in Chiang Mai, so that the two of you can both move the mouse. In this case, either ask the TO to find out who it is and ask them to kill their vnc session, or open a text editor on the DRPC that everyone would be able to see and type, for example, Vik Dhillon here. I am using ULTRASPEC now. Please could you disconnect your vnc session? Thanks.

If the DRPC physical keyboard has numlock or caps-lock enabled, vnc will also use this, so ask the TO to press the relevant button on the DRPC to turn it off.

Finally, it is possible someone has switched desktops by accidentally hitting ctrl-alt-f2, for example. This would make vnc appear as a black screen with a login prompt at the top. To fix this, ask the TO to type ctrl-alt-f7 on the DRPC in the control room, which will hopefully bring the desktop back.

everything is running really slowly, or freezes completely

The status monitor can slow things down, so make sure that it is not running by typing killall gkrellm.

If the link freezes completely, it could be because the vnc connection has gone down. Find the terminal in which you made the ssh tunnel (on the Warwick PC this can be found by clicking "activities" in the top-left of the Warwick PC, and raising the original terminal where you typed "tunnel"). The terminal may show that the connection has dropped. If so, try to re-establish the connection by re-opening the ssh tunnel. It may say "connection to port 40002 (or 50002 at Warwick) refused", or similar, a few times, but persist, because it can be a little slow to re-establish the link. Any run you have going will keep running, and you may have to ask the TO to stop the run directly on the control-room PC if you can't get things going again. Note that the software on the DRPC will automatically attempt to re-open the incoming tunnel to your local machine.

reduce light-curve plot corrupted

This could be due to a change to ignore bad times for hipercam in the pipeline, which has a bad effect for ULTRASPEC. You may have to start with first=5 to get past riduculous times in the past. Tom hopes to fix this at some point.

Really high readout noise

Sometimes the readout noise can be extremely high, so high in fact that a diagonal pattern can even be seen in the flat fields. This is possibly because the TO has forgotten to turn off the air conditioning in the dome. Ask them to do so, even if they do complain that the humidity is high (as there is little point in air conditioning the dome when it is open!).


Contacts

If you experience problems with remote observing with ULTRASPEC, please email/phone one of the contacts.