SCT White Board

Quick Links: Warm Start DAQ DCS DQM Problems and Solutions Cooling Heater Pads ID Environment Cooling Failure Power cut Recovery IDE white board

SCT Run Coordination: 163738

SCT ATLAS Control Room: 71344

SCT Satellite Control Room: 70946

On-Call Experts

When in doubt always call the Run Coordination phone. If you are experiencing a specific problem, you may call the system expert (full list on SCT phones).

System On-call Number
DAQ & DCS 162034
Monitoring 160037
IDGen, including cooling 162449
FSI 160609

Please re-read the shifter guide if this is the first of a block of shifts.

Latest SCT News

Please read ALL items below at the start of your shift. One or more items further down the list might have changed.

  • NB Normally ROD removal - recovery works
  • TTC restart is only for extreme situations

See below for news from run 1.

  • We run in STBY at all times (all modules with HV at 50V) unless the Stable Beam flag is present. If (and only if) stable beams are declared by the LHC, and with the agreement of the shift leader, you may ramp the SCT to READY (150V) by following the instructions at SCTWarmStart.

Useful Information (probably won't change very often)

  • The module naming conversion database is available here

  • For DAQ or DCS issues. please call the 'DAQ & DCS' on-call phone. If the problem is more difficult, the DAQ & DCS on-call person can call the appropriate 'Super Expert' on-call phone.

  • If for some reason you have to log in to the SCT computer in ACR (after a reboot or some other problem), use the username crsct without password. Do not use your personal account.

  • The problems database has moved and is now found under the ID->Inner Detector->ID operations DB menu item.For any issues, problems you encounter, or questions you may have, please first look in the operations (problem) database to see if there is a solution, before consulting the other documentation. The problem database is up to date, other documentation might be obsolete. If you can't see guidance in the operations database, look in the Quick links.

  • For link errors, please use the new Link Error Monitor which automatically displays link errors and attempts to recover modules without the manual intervention of the shifter. Read about the new monitor here. The old SctGUI ROS monitoring display is still available, but you should use the link error monitor instead.

Latest SCT News (needs review for run 2)

Please read ALL items below at the start of your shift. One or more items further down the list might have changed.

Last updated 07-06-2012

  • SCT end-caps should no longer be put to SUPERSAFE after beam dump - the Standby voltage for some endcap voltage has been changed to 20V, so just leave the full SCT in STANDBY between fills.

  • The gatherer problem has been solved. The (in)efficiency plots are trustable now. Please report any issues you see in your shift summary! (21-09-2011)

  • The SCT will now automatically switch on after declaration of stable beams, provided all indicators in the 'ip collimator check' panel are green (07-09-2011)

  • The SCT DQ Checklist is not updated to the new OHP and DQMD configuration yet (the ID DQ checklist is). Ignore histogram checks for histograms that have been removed! Note: in OHP histograms related to tracks have been moved to "IDGlobal" and the ROS monitoring histograms are located in "Beam" now. (17-08-2011)

  • OHP has been changed to the combined ID configuration, see new setting for DAQ panel below. Please report if you have any problems or if you have suggestions! (28-07-2011)

  • Heater pad ENDCAP/C/OTE_M_05/1 has been oscillating since some time, leading to frequent temperature low warnings as it goes just under 15 degrees C. Investigation and repair in the next days (as per 12-07-2011).

  • If you are not using the new IGUI panel, keep the Link Error Monitor visible at all times, and glance at it frequently. If there are too many errors (top of monitor is red) then take action.

  • If a ROD appears GREY in the TTC panel, check with the run control shifter if a ROD has been removed. If he/she says no, then check the link error count - if this is a 'normal' number, ie less than 30, then there is no need to take any action. There is an intermittent fault with the TIM in this crate which makes the TTC gui report incorrectly that a ROD has been stoplessly removed. Make a note in the elog and/or call the DAQ expert (in normal working hours).

  • Reminder of TTC panel colours:
    • GREEN means OK, no action needed
    • ORANGE means OK, no action needed (orange means was busy, but not busy now, probably due to link being masked at time of TX death)
    • RED means rod is BUSY, take action.
    • GREY means the rod has been stoplessly removed - take action.

  • SCT has asserted BUSY more frequently recently. If SCT goes busy, it is important to act quickly. The run control shifter should disable the ROD immediately, and then you should immediately follow the instructions to recover the ROD in the operations database shown here.

  • Please check the 'SCT Occup' indicators in the IP Collimator Check panel of the ID FSM well before stable beams are declared to make sure they are not red (red might mean that the ros monitoring applications have crashed). Check the indicators when LHC goes to SQUEEZE, and if red, check the ROS monitoring applications are not ABSENT in the run control tree - if they are, ask the run control shifter to restart them.

  • Recently, the monitoring has had some problems with hot modules. If you spot a hot module (a module with an order of magnitude more hits than the modules around it), please follow the instructions on: SCTMonitoringShifter#Hot_module_recovery

  • Be aware that BCM and IDGEN are running on the same computer (pc-atlas-cr-19). If they have a problem and restart or kill the X server, this will also affect you. Better do not use it for shift notes!

  • If LHC needs to make some adjustments when in stable beam, they remove the STABLE BEAM flag and move to ADJUST. In preparation for this move, the LHC handshake mechanism moves first to WARNING and then to IMMINENT. At WARNING, the SCT will ramp down to STANDY automatically, so, from now on, the shifter needs not to manually ramp to STANDBY when the LHC handshake goes to WARNING (the shifter leader should inform anyway the SCT shift when this happens).

  • If an unplanned beam dump occurs during stable beam, normally the Stable Beam flag is kept until LHC starts ramping down, typically for a time of the order of a few minutes. The shifter should calmly ramp down the SCT by sending the FSM command "Go To Stand-by". This manual operation is useful to disentangle any trip or dump-related issues. It is necessary to ramp down HV in order to be ready for the next injection. In any case, the subsequent loss of stable beam flag will automatically ramp down the HV. However, in case of a cryo problem (quench or "lost conditions") the stable beam flag is lost at the very same time as the beams are lost.

  • If ATLAS decides that there is time available for sub-detectors to do calibration, call the DAQ expert on call to tell them that there is time for calibration. Only do this between the hours of 0700 and 2200 (ie, not during the night). The expert might decide to run the SCT in a standalone partition either from the ACR or IDSCR, but only after informing the shifter in the ACR. If the SCT has to remain in standby, we can still do opto calibrations.

  • In special circumstances with no beam present (eg no beam expected for many hours), we may run with the SCT 'READY' (HV at 150V). Call the SCT run coordination to get a confirmation! They need to disable the SCT beam protection first (which prevents us from switching to 'READY' state in the absence of the stable beam flag from LHC).

  • Please note, issues relating to BCM or BLM are NOT the responsibility of the SCT shifter. The pixel ACR shift takes care of those.

Run Coordination Plan of the Day



Last runs in the ATLAS partition



Last page update 27 Feb 2021 07:25:51

DAQ Panel Configuration for ATLAS

Check these values in the corresponding setup items of the DAQ Panel:

Version tdaq-05-05-00
Setup Script /det/tdaq/scripts/setup_TDAQ_tdaq-07-01-00.sh
Partition Name ATLAS
Database File /atlas/oks/tdaq-07-01-00/combined/partitions/ATLAS.data.xml
MRS Filter SCT
OHP Opt ** NEEDS UPDATE **
Cosmic OHP ** NEEDS UPDATE **

For standalone running, the DAQPanel settings are defined here.

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf IDPanelTutorial.pdf r1 manage 667.2 K 2017-07-27 - 14:54 UnknownUser ID panel tutorial
Edit | Attach | Watch | Print version | History: r244 < r243 < r242 < r241 < r240 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r244 - 2020-10-19 - mmorinag_40CERN_2eCH
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ATLAS?Please contact the page author (see Topic revision above) or the Run Coordinator of the specific system.
Contact SysAdmins support only for technical issues