ULTRACAM user manual
Version: 22 May 2003
- Introduction
- Powering up
- Powering down
- Afternoon activities
- Taking data
- Looking at data
- Archiving data
- Changing CCD parameters
- Setting up on an object
- Drift mode
- Calibration frames
- AutoLogger
- Changing filters
- Before you go to bed
- Troubleshooting
- Contacts
Introduction
This manual describes how to observe with ULTRACAM. There are four other
documents which should be consulted if the information you require cannot
be found here:
- ULTRACAM
system maintenance document. This document describes ULTRACAM's
camera head handling and system maintenance procedures. It should only
be of relevance to project personnel.
- CCD
application document. This document describes the various CCD
controller applications that can be run with ULTRACAM. You should
regard the applications document as the primary source of reference
for the different readout modes and their associated parameters,
although a summary of its contents is given below.
- ULTRACAM
pipeline software manual. This document describes how to use Tom
Marsh's pipeline data reduction software, which is the best way of
viewing the data whilst observing.
-
Telescope operators guide. This document describes how to use ULTRACAM
with the WHT Telescope Control System and is intended primarily for telescope
operators at the ING..
Powering up
A description of how ULTRACAM's cables and pipes are connected is
outside the scope of this manual and it will be assumed that this
procedure has already been successfully accomplished. To then start
ULTRACAM from cold, the following operations must be performed:
- Check that the ULTRACAM network switch housed in the blue
box near the data reduction PC is powered on.
- Turn on the data reduction PC, housed in the aluminium case, by
flicking the switch at the rear of the case and then pressing
the round button on the front panel.
- Check that the red light on the mains socket panel at the top of the
rear of the electronics rack is illuminated - this means that the rack is
connected to the mains.
- Check that the GPS system is powered up and working by looking for the
flashing green LED on the GPS electronics box, which is located at the front
of the top shelf of the rack. If it is not flashing, check that the
power supply for the GPS is connected at the rear of the electronics box.
- Turn on the SDSU controller by flicking the switch marked "SDSU
Power Supply" from 0 to 1. The switch can be found approximately half way
up the rear of the rack near the centre of a white panel. If
successful, two LED's should illuminate on the SDSU controller.
- Turn on the rack PC by flicking the switch marked "Power" under
the key-operated flap on the front of the PC. Booting takes around one
minute if the system was shut down cleanly. If the PC is recovering
from a crash, rebooting may take up to 5 minutes whilst the system
checks the disks. If the PC fails to boot, you will have to connect a
monitor to the PC and inspect the terminal output for clues as to what
might be going wrong. (Note that there is a keyboard permanently
connected to the rack PC which lives on the top shelf of the rack
unit).
- Don't flick any of the switches described below
until you have read the whole of items 7 and 8. Turn on the
peltier temperature controllers by flicking the two switches at the
rear of the rack from 0 to 1. The switches can be found approximately
half way up the rear of the rack, on the right-hand side of the white
panels next to where the mains cables plug in. When you have done
this, immediately check the temperature displays on the front
of the rack - they should not be on. This is because, to prevent
thermal damage to the CCDs, the power to the peltiers will not turn on
if there is no water flow. Likewise, if the water flow is suddenly
cut off, the peltier power will automatically switch off. If the
peltier displays do come on, immediately raise the set temperatures to
the current chip temperature (see below).
- Turn on the water chiller by flicking the switch at the top of the
rear of the unit and then pressing the right-hand button on the front
panel with the "|/O" on it. The chiller will spring into life and
indicate the current temperature of the water, which will begin to
drop as the chiller starts cooling the water down to the set
temperature. To change the set temperature, press the left-hand button
with the penannular arrow on it and then alter the set temperature
with the up and down arrow buttons. Press the button with the
penannular arrow on it again to complete the setting process.
As soon as the water chiller comes on, the peltier displays on the
front of the rack will illuminate. There is one display for each CCD
head, although which display corresponds to which head can only be
determined by tracing the cables from each CCD head to the back of the
rack unit. The upper figure on the display shows the current CCD
temperature and the lower figure shows the set temperature. The
temperature controllers will immediately begin cooling the chips to
the set temperature by running the peltiers at their maximum power of
-55%. However, it is possible to cause thermal damage to the chips by
cooling them by more than 5 degrees per minute, so it is essential to
immediately increase the set point to 5 degrees below the
current chip temperatures using the up and down arrow buttons to the
right of the temperature displays. Once attained, drop the set
temperatures by approximately 5 degrees per minute until the desired
temperature of -40 degrees is attained. Once the chip temperatures
have stabilised at -40 degrees, check the peltier power levels by
pressing the left-hand turquoise buttons on the temperature displays
once. The power level should be approximately -30%. If the value is
close to or equal to the maximum value of -55% then the heads may have
lost their vacuum and need to be pumped down. You can return to the
temperature display by pressing on the black button in the middle.
Powering down
During an observing run it is normal to leave the water chiller, peltier
devices and SDSU controller running continuously. At the end of a run,
however, the system needs to be powered down in the following order:
- Turn off the SDSU controller (refer to the Powering
up section for the location of all relevant power switches).
- Turn off the water chiller. This will cut the water flow and hence
will automatically turn the peltiers off (but check the peltier displays
to make sure). With all cooling turned off, the chips will gradually
warm up to ambient temperature at a safe rate.
- Turn off the peltier devices.
- Shut down the rack PC by typing "init 0" in an xterminal connected to
the PC and then wait for a couple of minutes. This does not turn the PC
power off, so when the system is halted it will then be necessary to press
the "Power" button on the front-panel of the rack PC.
- Turn off power to the rack unit.
- Shut down the data reduction PC by pressing the ctrl-alt-del
buttons at the login prompt. The PC will power down automatically on
completion.
Afternoon activities
Before you start observing in the evening, you should perform the
following system checks:
- Check water temperature on the chiller display is set to 10 degrees,
unless this is within 10 degrees of the dew point (in which case the
water temperature should be increased). The dew point can be determined
from the observatory meteorological system. A useful check on this is to run
your hand along the base of the SDSU controller. If it feels even slightly
clammy to the touch, increase the water temperature slightly.
- Check CCD temperatures on the rack-mounted temperature controllers are
stable at -40 degrees.
- Check peltier power usage on the rack-mounted temperature controllers.
These should typically be around -30%. If the value is close to or equal
to the maximum value of -55% then there is either a problem with the
vacuum in the head or the CCD temperatures have been set too low (or
the water temperature too high).
- Check for water leaks by running your fingers along the pipe connectors
on the water chiller, the SDSU controller and the individual CCD heads.
- Check the nitrogen gas flushing system is operable by ensuring that
all the blue pipework and syringe needles on ULTRACAM are connected, that the
small brass valves on ULTRACAM are fully open, and that the gauge feeding
ULTRACAM is set to a flow rate of appoximately 1 litre/minute.
- Ensure that the water pipes and GPS cable leading up to the instrument
are neatly arranged beneath it and are in no danger of snagging on any
equipment.
- Ensure that the star-field tube is not mounted on the ULTRACAM collimator,
that the focal-plane mask is fully retracted and its foam baffle is in
position.
- Check which filters are installed in front of each CCD, either directly
by taking the cartridges out or indirectly by looking in the filter case
to see which filters aren't mounted (see Changing filters).
- Start the observing system (see Taking
data), take some bias frames and check the readout noise and
bias level (see Calibration frames
and Looking at data).
- Ensure the safety of data taken the previous night by moving it to a
subdirectory on the rack PC, copying it to the data reduction PC and
archiving it to DVD or DDS4 (see Archiving
data).
- If you want the first run of the coming night to be run001 (recommended),
make sure all runs in /data on the rack PC have been moved/deleted and then
restart the observing system (by hitting ctrl-c in the camera and
filesave windows and typing simple_startup -full name again - see Taking data).
Taking data
ULTRACAM can be controlled from any linux PC connected to the ULTRACAM
internal network (see the Troubleshooting
section for details), although it is usual to use the aluminium data
reduction PC for this purpose, as described below:
- Log into the ultracam data reduction PC (known as ultracam).
If you don't have a user account on this system, please
contact Paul Kerry.
- Log into the rack PC (known as ucam2) as root. It is only
possible to do this via ultracam using ssh. To ease this process, you
can type rack, which will automatically log you into ucam2 as
root without having to type a password. If you are asked for a
password, then a setting is amiss - please contact Paul Kerry. Because you are
logged in as root you need to be extra-cautious with your typing.
- In an xterminal on ucam2, type:
ucam_startup
and follow the prompts. This script sets the system clock, which is often
very inaccurate, leading to confusion with data file-creation times. The
script also resets the system logging files, which become full with ULTRACAM
debugging messages, causing the rack PC to run extremely slowly. Once run,
you shouldn't need to run this script again until the next night (or the next
time the rack PC is rebooted).
- Ensuring that the SDSU is switched on, type the
following in an xterminal on ucam2:
simple_startup -full name
where "name" can be any string you choose (and does not have to be
surrounded in quotation marks if there are no spaces in it). The
"-full" option tells the system that you don't intend to use drift mode.
If you wish to use drift mode, you need to use the "-fast" option
(see the Drift mode section below). The
following windows should then appear:
The "Camera" window (top-left) provides information on the commands
used to control the CCD cameras, which are sent to the SDSU controller.
The "Filesave" window (top-right) provides information on the commands
used to define the quantity of data to be expected, and are sent to the
SDSU-PCI card in the rack PC. The "Netscape" window (or "WebGUI") sends
the xml documents
containing the camera and filesave parameters to the SDSU controller
and PCI card via the http protocol.
- Set the file headers by clicking on the following links in the
"engineering" column of the netscape window:
- setup wht (camera)
- setup wht (filesave)
- setup ultracam (camera)
- setup ultracam (filesave)
These links must be clicked in order from top to bottom and
each time you click on a link in the web browser you must wait for the
netscape "busy" graphic to stop before clicking on the next link. If each
operation is successful you will get a message in the status window at
the bottom left of the web browser which contains the word "ok". You will
also obtain a line in the history window at the bottom right of the web
browser showing you which links have already been activated.
- Power up the SDSU controller by clicking on the following links in
the netscape window:
- poweron (camera)
- poweron (filesave)
- GO
You should now see a full set of LED's illuminated on the SDSU controller.
Because of a bug in the current ULTRACAM software, it is now necessary to
do the following to complete the SDSU power-on sequence:
- Press ctrl-c in the camera window.
- Press ctrl-c in the filesave window.
It is essential that you do this in the order given above. There is no
need to close the netscape window as it will respawn itself automatically.
- Now restart the software using the command:
simple_startup -full name2
where "name2" can be any string as long as it is different to the string
you used the first time. The camera and filesave windows will now
reappear and, after a few seconds, the netscape window will be respawned - make
sure you wait for this to happen before clicking on any links.
- Set the file headers again by clicking on the following links in the
"engineering" column of the netscape window, in the following order:
- setup wht (camera)
- setup wht (filesave)
- setup ultracam (camera)
- setup ultracam (filesave)
There is no need to click on the "poweron" links again, as the SDSU controller
has already been powered on.
- You are now ready to take data. You can select either one of the
"generic" modes or one of the "user-defined" modes - the difference
between these two options is described in the section on Changing CCD parameters. If, for
example, you decide you want to take data in 2-windowed mode using the
CCD parameters given in the generic xml file, you should click on the
following links:
- 2 windows (camera)
- 2 windows (filesave)
- GO
- STOP
It is always necessary to download first the camera application and
then the filesave application. You must wait for each application to
successfully download before clicking on the next link. In the above
example, the NO_EXPOSURES parameter is set to -1 in the xml file (see
Changing CCD parameters), and
hence data will be taken indefinitely until you click on the "STOP"
link. When this occurs, you will obtain a message in the filesave
window telling you how many exposures were taken and a message in the
status window saying "ok".
- Once you have started a run, type the following command in an xterminal
logged into the rack PC:
ucam_data_limit
Hit return in response to all the prompts (unless you want to check the
speaker volume or change the file size limit). This script checks the
latest run and outputs an audible warning when it becomes dangerously
close to the maximum file size limit of approximately 2 GBytes.
When you hear the alarm, you must stop the current
exposure. If the file size goes over the limit, the rack PC will
crash.
- When you are ready to take your next exposure, you must reload the
camera and filesave applications for your chosen CCD setup and then
click "GO", even if you have not changed any of the parameters.
Looking at data
You can use the ULTRACAM pipeline reduction software to look at data
either in real-time or off-line - see the manual for
details. The latest version of the software is version 1.1
and can be initialized by typing the following on the data reduction
PC:
source /home/trm/.cshrc
ultracam
If you have problems running this version, you can fall back to
version 1.0 (installed on Wed Jan 15 16:35:55
GMT 2003) by typing the following on the data reduction PC:
source /usr/local/ultracam/trm-ultracam-1.0/Ultracam
You can then type any of the pipeline reduction commands, e.g.
rtplot. In order to run the system whilst observing, it
is necessary to access the data on the rack PC over the server. To
do this, open an xterminal on the rack PC and type:
FileServer
Archiving data
Please refer to Paul Kerry's guide to
archiving ULTRACAM data.
Changing CCD parameters
ULTRACAM can be used in 7 different observing modes:
- Full frame with chip clearing.
- Full frame with no chip clearing.
- Full frame with clearing and overscan.
- Two-windowed mode.
- Four-windowed mode.
- Six-windowed mode.
- Drift mode.
Each of these modes are described in much greater detail in the CCD application document.. Once you have decided
which mode you want to use, place your mouse over the relevant option in
the "generic modes" column of the netscape window to determine the name of
the xml file which you will need to edit to change the CCD parameters.
Now open a window on the rack PC and type the following command:
cd /ultracam/config/xml
Be very careful in this directory, as it is the main
repository for the xml files used to set the ULTRACAM CCD parameters.
If you do delete anything by accident, there is a copy of the directory
on ucam2 in /root/pkerry/xml_backups_18_5_03.
You can now edit the required xml document
(e.g. set_ap3_fullframe.xml) and change any of the following
parameters:
- NO_EXPOSURES - the number of exposures. It is usual to set this to
"-1", which means that exposures are obtained until the "STOP" link is
clicked on the netscape window.
- GAIN_SPEED - the gain and readout speed. It is usual to set this
to "0xcdd", which is a gain of 1.2 electrons/ADU and a slow readout
speed (10 microseconds/pixel/second/channel). Setting to "0xfdd" gives
a faster readout speed at the expense of significant pickup noise on the chip.
- EXPOSE_TIME - this is the exposure delay time (in units of milliseconds)
which should be added to the irreducible minimum exposure time due to the
time it takes to read the chip out to give the total exposure time.
Do not enter a value of 0 (or less), as it will crash the
system.
- X_BIN_FAC - the x-binning factor, which must be an integer multiple of
the x-size.
- Y_BIN_FAC - the y-binning factor, which must be an integer multiple of
the y-size.
- V_FT_CLK - the vclock speed. It is usual to set this to "0x990000",
which gives a vertical clocking speed of 24 microseconds/row. If you lower
this, you run the risk of lowering the count-level at which the "peppering
effect" occurs (see the Troubleshooting
section for details).
- X1L_START - the x-start position of the first window on the left-hand
side of the chip.
- X1R_START - the x-start position of the first window on the right-hand
side of the chip.
- Y1_START - the y-start position of the first pair of windows.
- X1_SIZE - the x-size of the first pair of windows.
- Y1_SIZE - the y-size of the first pair of windows.
- X2... ditto for the second pair of windows.
- X3... ditto for the third pair of windows.
For detailed information on the allowed ranges of each of the above parameters,
you should refer to the CCD application document.
Once you have set your desired parameters, save the file and then download
the application by clicking on the appropriate links in the netscape window.
For setups which you will reuse during the run, it is much easier to define
your own "user-defined mode". To do this, save the xml file you have been
editing as:
set_objectname.xml
objectname can be any string, but most usually is the name of
your target. Then, type the following commands on the rack PC:
cd /ultracam/src/ucam/WebGUI/vik
emacs control.html
At the end of this file you will find an html list composed of a number of
the following items:
<li> <a href="javascript:cam('set_objectname.xml')">
objectname (camera)</a> </li>
<li> <a href="javascript:fs('set_objectname.xml')">
objectname (filesave)</a> </li>
Change the "objectname" string to whatever you have decided to call your
xml file, save the file, and then press the "reload" button on the netscape
window. Your new setup should now appear as an option under the "user-defined
modes" column. Be careful in this directory, as it is the main
repository for the html files used for the WebGUI (i.e. the netscape window
on which you run the observing system). If you do delete anything by
accident, there is a copy of the directory on ucam2 in
/root/pkerry/webgui_backups_21_5_03.
Setting up on an object
Setting up on an object can be a time-consuming process with
ULTRACAM. The following tips will help to cut the acquisition time.
Focussing
The WHT focus with ULTRACAM is very stable and always close to
98.00mm. Therefore, don't waste time at the start of the night
slewing to a focus star and focussing the telescope on it. Instead,
slew to your first science target, set the required CCD parameters
and start taking proper data. Then, using the "Gauss" option in
"rtplot" (see Looking at data) to
determine the stellar FWHM, ask the telescope operator to change the
telescope focus in steps of 0.05 or 0.1mm and minimise the FWHM. These
are tiny increments which won't affect the quality of your data
significantly and, because you are acquiring science data whilst
focussing, you don't have to rush the process.
Target acquisition
The WHT pointing calibration ("CALIBRATE") need only be done once, on
the first night ULTRACAM is mounted on the telescope. Thereafter, it
is sufficient to reload this calibration every night, saving a
considerable amount of telescope time during evening twilight.
The WHT points to better than a few arcseconds and the centres of the
ULTRACAM chips are within a few arcseconds of the rotator centre (to
which the WHT points). Hence, as long as your coordinates are accurate
to a few arcseconds, you can be certain that when you slew to your
target it will appear at the very centre of the ULTRACAM chips, making
target identification very easy.
The scale of the ULTRACAM chips on the WHT is 5 arcminutes (0.3
arcseconds/pixel), but the orientation is variable, as it depends on
the orientation of ULTRACAM, its mounting collar and the A&G box with
respect to the WHT, all of which are subject to change from one
observing run to the next.
The best way of positioning targets on the ULTRACAM detectors is to use
the "setup" application under the "generic modes" column on the netscape
window. This is just a two-windowed setup in which the windows
are the same size as the whole chip and the pixels are binned by a
factor of 4. With a 1 millisecond exposure delay, this results in an
exposure time of approximately 0.25 seconds, with negligible
dead-time. The rapid readout and large field of view results in an
acquisition-TV-type mode, which greatly facilitates target acquisition.
Setting CCD windows
The "rtplot" program has an option in which it is possible to display
the outlines of CCD windows superimposed on the image of the field.
The CCD windows which are plotted are defined in a file, which can be
edited in real-time to adjust the positions and sizes of the windows
on the display. The final window parameters can then be copied from
the window file into the appropriate xml document. For details of the
format of the window file, see the ULTRACAM pipeline
software manual.
Exposure times
The "Gauss" option in "rtplot" can be used to determine the maximum
counts in an aperture centred on the star. Do not
mistake this with the "peak height" value - the maximum value of the
Gaussian fit to the stellar profile - which may underestimate the true
maximum. If the exposure time needs to be lengthened, increase
the "EXPOSE_TIME" parameter in the appropriate xml document (see Changing CCD parameters). If the
exposure time needs to be decreased, and the EXPOSE_TIME parameter is
already at its minimum value of "1", you will need to either decrease
the size and/or number of windows being read out, bin the pixels,
change the "GAIN_SPEED" parameter or adjust the window positions to
minimise the "DIFF_SHIFT" (see the CCD
application document).
Drift mode
To obtain the highest frame rates it is necessary to use drift mode,
where CCD windows are stacked up in the masked region of the frame
transfer chip. A full description of the algorithm is given in the CCD
application document. To take data in drift mode it is necessary to
follow exactly the same procedure as described in
Taking data, except that the "-fast" option
must be used instead of "-full" as follows:
simple_startup -fast name2
The "-fast" option must only be used when running drift mode
applications - the system will not work if you attempt to run any of
the other observing modes. Likewise the system will not work if you
attempt to run drift mode applications having brought the system up
using the "-full" option.
It should be noted that with the current version of the ULTRACAM software
it can take over a minute to download drift mode applications, i.e. the
netscape window will be busy for over a minute before responding with
an "ok" in the status window. On no account should any other links be
activated before the browser has successfully downloaded the appplication.
Calibration frames
Taking calibration frames for ULTRACAM is a great deal easier than with
conventional CCD detectors thanks to the high speed readout. Dark current
is less than 0.1 electrons/pixel/second, a factor of approximately 50
less than the photon rate from the u'-band sky on the WHT, which means
that dark current is a negligible source of noise with ULTRACAM. Coupled
with the fact that it is difficult to remove all sources of light in a
telescope dome, and hence difficult to obtain accurate dark frames, only
flat-fields, bias frames and standard stars are discussed below.
Flat fields
It is best to obtain flat fields of the twilight sky. Ask the telescope
operator to point to a blank field in the east during evening twilight.
Once there, start spiralling the telescope to move any faint stars present
in the field so that they are removed when median-combining. To make the
telescope spiral, enter the following command on the ING instrument control
computer:
load [.vsd]spiral.icl
spiral
You will then be asked to enter the step length in arcseconds, the
number of steps and the number of seconds to pause between each
telescope slew. The step length should be much larger than a stellar
diameter, but not so large that the spiral rapidly moves the telescope
away from the blank field (and introduces any bright stars in the
periphery of the field). A value of 10 arcseconds seems to work well,
in conjunction with 100 steps and a pause of 7 seconds (the time it
takes to read out a full-frame image with the slow readout speed
setting). When you wish to cancel the spiral and return the telescope
to its nominal position, type ctrl-c at the ICL prompt and
ask the telescope operator to type "n" on the TCS.
Unless you know exactly what window setup you are going to be using,
it is best to obtain full-frame flats, as the pipeline reduction system
can crop them to match any window setup. Set the exposure time to at
least 1 second, which is approximately 40 times longer than it takes
to perform one frame transfer; this minimises the error due to each
pixel accumulating charge as it vertically clocks through the image
region.
Bias frames
Again, these are best obtained in full frame mode. Set the exposure
time in the xml file to its minimum value of 1 millisecond (see Changing CCD parameters.) Ensure
that it is completely dark in the dome and, if time permits, place the
A&G-box comparison lamp mirror in the beam to minimise the light
falling on the detectors; the command AGCOMP typed on the ING
instrument control computer moves the mirror into the beam and
AGMIRROR OUT moves it out again. You should find that the
bias level and readout noise on all three chips is approximately 2000
and 3.0-3.5 counts, respectively.
Standard stars
These are best selected from the list of Smith et al. 2002 (AJ, 123, 2121).
AutoLogger
The AutoLogger is a c-shell script which produces a log of
ULTRACAM observations on a web browser. The script can be either run
in real time, in which case it must be run on the rack PC, or
off-line, in which case it can be run on either the rack PC or the
data reduction PC. The script works by polling the directory
containing the data and extracting information from all the xml files
it finds. It also determines the time and size of each ULTRACAM data
file and uses the latter information to calculate the number of frames
in each run. The object name, filters and comments on each run are
input using an optional comments file, which must reside in the same
directory in which AutoLogger is run. An example of the optional
comments file can be found here - it is
essential that you do not change the format of the file, i.e. the two
header lines and the column positions. Also, do not use any
characters in the file which are not recognised by html - if for some
reason the comments or object/filter names you have typed in do not
appear on the web page, then it is likely you have used an illegal
character. Finally, don't leave the object or filter columns blank -
if you want the log to show a blank entry in these columns, enter
"nbsp&" in the appropriate position in the optional comments file.
To run the AutoLogger for data on the rack PC whilst observing on 20/05/2003,
type the following:
cd /ultracam/src/ucam/WebGUI/vik/AutoLogger
emacs 20_05_03_log.dat - and enter the object name, filters and comments for each run
AutoLogger - ignore the various error windows which will pop up
> /data
> 20_05_03_log
The script will run indefinitely, polling the data directory a few
times every minute and looking for changes in either the data files or
the comments file. If it finds a change, it will update the log
displayed on the web browser. To exit AutoLogger, just type
crtl-c, but only do this when AutoLogger says that it is safe
to. The final log is written to a file in html format - in the example
above the resulting file would be called 20_05_03_log.html. To print out the html
log file, it is best to run the konqueror web browser on the
data reduction PC, load the html log, and then print to a postscript
file, setting the page size to A4 and the orientation to landscape.
AutoLogger can also be run off-line, either on the rack PC in exactly the
same way as described above, except that any directory containing ULTRACAM
data can be polled, or on the data reduction PC. In the latter case,
AutoLogger resides in /home/vsd/AutoLogger and it can be used to
poll any directory on the data reduction PC which contains ULTRACAM data.
As well as the automatic log, we also keep a hand-written log which
details weather conditions, run numbers, filters and any comments. It
is essential that this log is carefully kept as it provides a backup
to the electronic version. Before you start observing on each night,
print out one copy of this title page template and then
multiple copies of these blank tables.
Changing filters
There are currently 8 different filters available for use in ULTRACAM:
u', g', r', i', z' and 3 "clear" filters. There are also 3 spare
"windows" for the CCD heads, but these cannot be used as filters
because they throw the instrument out of focus due to their different
optical thickness. The filters not being used are stored in a small,
blue briefcase. If you want to change a filter in ULTRACAM, you should
first mount the required filter in one of the two spare aluminium
cartridges, which you can also find in the blue briefcase. Make sure
that you use the plastic spacers to prevent the filter slopping around
in the cartridge. When ready to change the filter, carefully remove
the two syringe needles in the blue foam ring between the CCD window
and the filter holder on ULTRACAM and then slide out the cartridge
containing the filter you wish to remove. Using the 15cm metal ruler
found in the ULTRACAM tool box to prevent the blue foam ring from
slipping, slowly slide the new filter cartridge into the filter slot
(you may feel the ball-bearing retainer click into position when
complete). Slide the metal ruler out, remembering to replace the two
syringe needles. Don't worry if the blue foam ring is not centrally
located over the CCD window - it would have to be seriously out of
position to vignette the field.
Always use the optics handling equipment (e.g. latex gloves, lens
tissues, air spray) when changing filters - you can find these either
in the blue briefcase or the "optics handling equipment" box in one of
the ULTRACAM packing crates. When blowing air across the
filter, be sure to hold the can steady and upright, otherwise propellant
may fall on the filter. It is also wise to make a few test blows into
the air before spraying the filter.
Before you go to bed
- Make sure that the hand-written and automatic logs are up-to-date and
complete for the night.
- Shut down the observing system by typing ctrl-c in the
camera and filesave windows and closing the netscape window (see Taking data).
- Close down all windows which are logged onto the rack PC.
- Turn off the SDSU controller
(see Powering up).
- Copy all data obtained during the night from the rack PC to the
data reduction PC (see Archiving data).
- Go to bed.
Troubleshooting
Finger trouble
This is the most common cause of ULTRACAM crash and might be due, for
example, to clicking on the "GO" link on the WebGUI whilst an exposure
is still running. There is a faint chance that typing ctrl-c
in the camera and filesave windows and then retyping
simple_startup -full name might fix the problem. If this
doesn't work, close down all of the windows open on the rack PC except
for one, in which you should type "init 0" to shut the PC down. Go
into the dome and turn the power to the rack PC off (this ensures that
the SDSU-PCI card in the rack PC is properly reset). Turn the SDSU
controller off and then on again and then power up the rack PC.
If the rack PC hangs because of your finger trouble, and you are
unable to perform a clean shutdown are forced to turn the PC off,
then it is likely that the observing system will not work
properly when you bring it up again and you should perform a clean shutdown
and startup, as described above.
Data reduction PC hangs whilst exposing
If the data reduction PC hangs or crashes whilst exposing, it is
possible to safely stop the exposure by opening an xwindow on the rack
PC from another computer connected to the internal network (e.g. Vik's
lap-top) and then typing WebGUI. When the netscape window appears,
click on the "STOP" button to safely stop the exposure. With the data
safe, you can then try to reboot the data reduction PC. This will kill
the camera and filesave windows, of course, so you will also have to
reboot the rack PC.
There has been one occasion when the data reduction keyboard stopped
responding, probably due to an illegal combination of keystrokes. This
was fixed by killing the windows one by one until the offending window
had been killed and the keyboard started responding again.
WebGUI problems
- If the WebGUI stops responding when you click on the
links, try pressing the "Reload" button on the browser.
- If the AutoLogger suddenly appears on the WebGUI,
kill the AutoLogger script with ctrl-c and close
the AutoLogger browser. Then click the "Back" button on the WebGUI
to reinstate the instrument control page. You can then safely restart
the AutoLogger. Note that this problem tends to occur if you type
simple_startup -full name when the AutoLogger is already running
(e.g. following a crash).
- If the WebGUI disappears, but the camera and filesave windows are
still visible and taking data, type "WebGUI" in an xterminal on the
rack and the WebGUI should reappear.
WebGUI remains busy indefinitely when trying to power on SDSU
This is most probably due to the SDSU not being switched on. Once this
occurs, press the browser "STOP" button, ctrl-c out of the
camera and filesave windows and reboot the rack PC, remembering to
turn the SDSU controller on before you do this. If you can't reboot
the PC by typing "init 0" or ctrl-alt-del, you will have to
turn the power off on the front panel of the rack PC. The latter type of
nasty shutdown usually requires a further clean shutdown of the rack
PC before the observing system will start working again.
Exposures start, but can't be stopped
This is a classic symptom of GPS problems. When the GPS is not working
it is possible to start an exposure using the "GO" link on the WebGUI,
but when "STOP" is clicked the exposure does not finish cleanly
(either hanging or crashing). The resulting data file is usually
either corrupt or totally empty. Check that the GPS is working by
inspecting the flashing LED in the GPS box on the top shelf of the
rack, and also check the cable running from the telescope to the GPS aerial
outside. If there is high humidity outside, there is a possibility
that water has shorted the lightning surge protector on the GPS aerial
mast. If this has occured (which you can verify with a voltmeter),
bypass the surge protector by connecting the cable from the hose-reel
directly into the base of the GPS aerial.
An alternative reason for the above problem is if the system or kernel
logging daemons are stopped on the rack PC. To restart these, run the
ucam_startup on the rack PC.
High red-channel bias
This is a rare, intermittent fault which occurs when a new run is
started. The bias level on the red CCD suddenly jumps to a very high
level, sometimes with a loss of sensitivity and an increase in the
readout noise. It can usually be fixed by simply stopping the run and
restarting it. If this does not work, try it again, and again. If the
problem persists, try shutting down the rack PC, switching off the
SDSU controller and then restarting the system.
Networking problems
Please consult Paul Kerry's
networking document.
Add section on logging daemons, flow meter bypass, 50 mhz switch,
new turning on procedure, new power on procedure, looking at data
with martin's subroutine, looking at data with andy's subroutines,
inspecting GPS timings using Andy's routine, looking at log files with
period
Contacts
If you experience problems with ULTRACAM that you are unable to solve,
please contact one of the following project personnel:
- General problems: Vik Dhillon (vik.dhillon@shef.ac.uk,
work: 0114-222-4528, mobile: 07736647308)
- Mechanical problems: Mark Stevenson (m.j.stevenson@shef.ac.uk,
work: 0114-222-3504, mobile: 07919596693)
- Computing problems: Paul Kerry (star@shef.ac.uk,
work: 0114-222-3551, mobile: 07789143424)
- Pipeline data reduction software problems: Tom Marsh
(trm@astro.soton.ac.uk, work: 023-8059-2063)