This manual describes how to observe remotely with ULTRASPEC at the TNT. It is possible to do this from three different locations - the NARIT headquarters in Chiang Mai, Sheffield and Warwick. This document is an attempt to bring the following information into a single place:
This document does not give you checklists on what to do in the afternoon/start-of-night/end-of-night, nor does it tell you how to take bias frames, focus the telescope, take flat fields, or what to do when something goes wrong with the instrument. Please refer to the ULTRASPEC user manual for such information. Lots of useful links, e.g. to the blog, logs, filters, pipeline manual, phase II, etc, are given on the left-hand side of the ULTRASPEC homepage. A few additional links that may be useful whilst remote observing:
The section provides a checklist of things to remember when remote observing with ULTRASPEC on the TNT, whether that is from Sheffield, Warwick or Chiang Mai. For general checklists on what to do in the afternoon, start of night and end of night, please refer to the ULTRASPEC user manual.
The checklist below starts from a point where you have successfully followed your site-specific instructions (for Sheffield, Warwick or Chiang Mai) and you now see the DRPC desktop on your own local desktop:
eplanner.py targets.lis TNT targets.prg 2020-02-09 -p targets.pfx
WD0145+234 , 01:48:03.74 , +23:39:16.4
NLTT-11748 , 03:45:26.61 , +17:48:34.5
This section contains specific information for remote observing from Sheffield.
ssh -L 5900:localhost:5900 localhost -p 40002 -l observerYou will be prompted for the password of the observer account on the DRPC - ask a member of the ULTRACAM team if you don't know this. The above command creates a tunnel to the ULTRASPEC DRPC in the control room at the telescope. Leave this window running.
vncviewer -geometry 800x600 localhostYou will be prompted for the vnc password - ask a member of the ULTRACAM team if you don't know this. The above command opens a window on europa. If someone has already logged into the observer account on the DRPC in the telescope control room, you will be presented with the DRPC desktop, otherwise you will see the login screen of the DRPC (in which case you should log into the observer account as usual). It is recommended that you resize this vnc window so that it fills both monitors connected to europa.
This section contains specific information for remote observing from Warwick.
A remote control-room has been set up in the new NARIT headquarters building on the AstroPark. A new, dedicated 4MB/s link has been installed between the TNT and the NARIT HQ.
There is no accommodation at the AstroPark so you will need to stay at a hotel in Chiang Mai. Apichat usually makes the hotel bookings for us, and he tries to get hotels on the same side of the city as AstroPark to make travelling there easier. Vik recommends the Kantary Hills hotel on Nimman Road, which has a lovely pool. Dave recommends the IBIS Styles hotel on Cholprathan Road. Both are only a short walk from the Maya shopping centre where you can find many noodle bars for breakfast, and do shopping at the Rimping supermarket. Other hotel options are the Eastin Tan, Yesterday and Chiang Mai Orchid (in order of decreasing desirability).
The easiest way is to use a "Grab" taxi. Grab is the Thai equivalent of Uber. To avoid ridiculous call charges by your UK mobile phone provider, you will need a Thai SIM card. These can be easily obtained at either Bangkok or Chiang Mai airports, and most 7-Eleven shops sell them too, although some of the shops are only allowed to sell them during specific periods of the day. So if you can, get one at the airport. Make sure you make a note of your new Thai mobile phone number, because you will need to input it into the Grab app. Download the Grab app if you don't already have it on your phone. (Do this when on wifi at the hotel to avoid using up your data allowance!). Using the Grab app is easy, you can specify your destination either by typing in the name or on a map. I did have a couple of problems when the GPS location of the pick up point was slightly different to what the driver was seeing on his screen. However, the app shows you a map of where the taxi is, so I simply told the driver to wait where he was and I walked to him. The driver will not give you a receipt. You have to download them from the Grab website.
Ideally you should arrange for someone to meet you on your first afternoon to show you how to get to the control room (hereafter ACR) and where all the facilities are. Ask Apichat to nominate someone and arrange with them exactly where on site you should meet them. There is a security guard at the entrance to the park, just tell him you are going to the NARIT offices. You will be given a key to the door to the control room. On the day of your departure, this key has to be dropped into the 'Key-drop' box in the stairwell on the way down to the car park. Make sure they show you where the box is.
In the ACR, there is a kitchen with a fridge, a water dispenser and microwave. The kitchen has cutlery and crockery. You will need to bring your own tea/coffee and milk/sugar if you want hot drinks. And you will need to take your own food. If you are staying near the Maya shopping centre, then I suggest using the Rimping supermarket (see below) and the many noodle bars nearby. If you forget, and the hour is still reasonable you can get food delivered from https://www.foodpanda.co.th/city/chiang-mai.
If you're staying near the Nimman Road area, some suggested places to eat before/after you go to AstroPark are:
The ACR has a dedicated link to the TNT on Doi Inthanon. A photo of the room is shown below. There is the Ubuntu PC on the right, with two monitors, and a Windows PC on the left, which has two monitors on the desk and four mounted on the wall. Use the Ubuntu PC to connect to the DRPC and Rack. You can either SSH to the DRPC (ssh -X email@example.com) or you can connect using VNC. Connection using VNC is better if possible, since the graphical display will be much quicker. To connect using vncviewer, start it using "~/vncviewer" and enter the same IP address as above.
Contact a member of the Ultracam team if you have forgotten the passwords.
The windows PC can be used to display the interior and exterior webcams at the TNT, the TCS monitor, and a day-night map of the world which also has UT and LST showing. There are various icons on the primary monitor which open up these displays. They have to be manually dragged and dropped to their respective monitors. Normally the TO will remotely log in to this machine and set these displays up for you, as well as open up a skype connection to the telescope.
The Ubuntu PC has two monitors but also allows four workspaces so that gives a total of eight available screens, within which applications can be dragged and dropped. To enable multiple workspaces, you may need to click on 'Settings', 'Behaviour' and then select 'Enable Workspaces'. You can move between workspaces using ctrl + alt + arrow keys. As of Feb. 2019, we have started using VNC to remotely use the DRPC desktop. To start this, open a new terminal, and run ./vncviewer from the observer home directory. This way, inevitably kicking the plug from its socket under the table won't kill all your processes!
The best way of doing this is via skype on the windows PC in the RCR. First, get a skype account if you haven't got one. Then open skype by clicking the icon on the bottom bar of the desktop and log in. Then call 'TNO observer', ensuring that the speakers in the ACR are turned on. Another option is to use the telephone in the ACR. To call the TNT control room, press the speaker button and then type '611'. As a last resort, use a mobile phone (perhaps via skype to keep costs down). Make sure you get a list of the TO mobile phone numbers and email addresses when you arrive from a member of NARIT staff.
There is a printer in the ACR but it wasn't working when I was there. This made it more awkward to use finding charts and the eplanner. However, the finders can be emailed to the TO so it is not essential that the printer is working. I suggest you check it when you first get there whilst someone from NARIT is around.
The wifi in the ACR is good and the usual passwords work fine. There is also an ethernet connection. Beware of the extension lead under the desk. It is used to power all the PCs but the connector is not very secure and whilst I was unplugging my laptop charger it became dislodged and rebooted the PCs, right in the middle of the end-of-night tasks! So I 'borrowed' the extension lead from the TV room and plugged my laptop and phone charger into that.
This can be arranged with NARIT - it takes 1.5 hours to drive to the summit if there is no traffic. If you don't want to remotely observe, you can ask NARIT to book you a room in the Lodge at Doi Inthanon instead.
These are currently in the ULTRASPEC folders in the control room at the telescope. If you want them, you'll have to ask someone to bring them down from the mountain for you the following day.
The Windows PC controls the 4 wall-mounted screens, which display information on the current status of the telescope and have a default setup indicated by stickers attached to them. If it crashes you'll need to log in to the TCS to restore the displays; you may need to ask the TO for the password.
If you can't work for whatever reason, and if it isn't too late in the night, try the cinema on the 5th floor of the Maya shopping centre opposite the NARIT HQ. The films in English usually have Thai sub-titles rather than being dubbed, apparently. You could also try the famous Night Bazaar (http://www.visitchiangmai.com.au/night_bazaar.html) or the Sunday Market (http://www.visitchiangmai.com.au/sunday_market.html), which both run until 11pm-ish.
For problems specific to remote observing, please check the list below. For more general instrument problems, please refer to the troubleshooting section of the ULTRASPEC user manual.
The DRPC may not be booted or the external network could be down. You need to ask the TO to check the latter by logging in as the observer user and using firefox to browse to a site at Sheffield (or Warwick, if observing from there), e.g. http://www.vikdhillon.staff.shef.ac.uk. If this page opens, then the link from Sheffield/Warwick should be fine. Note that the tunnel software is constantly monitoring the connection and if it drops it will automatically try again forever without any user intervention - rebooting the DRPC just delays this happening. Hence only reboot the DRPC as a last report if nothing else works.
You may find that someone else is vnc'ed into the DRPC at the same time as you, e.g. from the control room in Chiang Mai, so that the two of you can both move the mouse. In this case, either ask the TO to find out who it is and ask them to kill their vnc session, or open a text editor on the DRPC that everyone would be able to see and type, for example, Vik Dhillon here. I am using ULTRASPEC now. Please could you disconnect your vnc session? Thanks.
If the DRPC physical keyboard has numlock or caps-lock enabled, vnc will also use this, so ask the TO to press the relevant button on the DRPC to turn it off.
Finally, it is possible someone has switched desktops by accidentally hitting ctrl-alt-f2, for example. This would make vnc appear as a black screen with a login prompt at the top. To fix this, ask the TO to type ctrl-alt-f7 on the DRPC in the control room, which will hopefully bring the desktop back.
The status monitor can slow things down, so make sure that it is not running by typing killall gkrellm.
If the link freezes completely, it could be because the vnc connection has gone down. Find the terminal in which you made the ssh tunnel (on the Warwick PC this can be found by clicking "activities" in the top-left of the Warwick PC, and raising the original terminal where you typed "tunnel"). The terminal may show that the connection has dropped. If so, try to re-establish the connection by re-opening the ssh tunnel. It may say "connection to port 40002 (or 50002 at Warwick) refused", or similar, a few times, but persist, because it can be a little slow to re-establish the link. Any run you have going will keep running, and you may have to ask the TO to stop the run directly on the control-room PC if you can't get things going again. Note that the software on the DRPC will automatically attempt to re-open the incoming tunnel to your local machine.
This could be due to a change to ignore bad times for hipercam in the pipeline, which has a bad effect for ULTRASPEC. You may have to start with first=5 to get past riduculous times in the past. Tom hopes to fix this at some point.
Sometimes the readout noise can be extremely high, so high in fact that a diagonal pattern can even be seen in the flat fields. This is possibly because the TO has forgotten to turn off the air conditioning in the dome. Ask them to do so, even if they do complain that the humidity is high (as there is little point in air conditioning the dome when it is open!).
If you experience problems with remote observing with ULTRASPEC, please email/phone one of the contacts.