TRT Shift Trainer Help

This area is intended to explain the shift Trainer's duties and to answer frequently asked questions.

TRT Shift Trainer's Duties

The TRT TTP are the TRT shift Training Personnel. There is a week long shift equivalent to 1 shift credit (if you think this needs to be increased, feel free to ask me to increase the credit). Most of the work currently is for supervising the practical sessions. To sign up, use OTP task 529095.
  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 2pm to 5pm in the ID satellite control room. Trainee shifters sign up to OTP task ID 30910. There are only 4 slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
  • Answer questions that the trainees have. New shifters may ask questions on the email list ( or at the practical sessions. Answer the questions to the best of your ability. If you encounter a question (or suggestion) that you cannot answer please post it the FAQs below and/or send an email to the ttp email list.
  • To try to schedule special sessions for shifters who cannot attend a regular session before their first shift. The extra sessions should be scheduled with the TTP who is on shift during the week if possible. If the on shift TTP cannot schedule a special session then they need to inform the other TTPs by emailing the TTP list and asking for a volunteer.
  • Make sure that documentation is up-to date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available here NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
  • THE FOLLOWING IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
  • Brand new shifters may also require an orientation session. This should be scheduled between the TTP on shift and the brand new shifter.

TRT Shift Trainer's FAQ's

Date Question Answer
15 Oct 2009 14:29 Where to find checklists filled out by previous shifters? There is a bug in the checklists mechanism that uploads the checklists to the elog. This is to be Followed up by Jim D. Usually once checklists are filled out they should go to the elog automatically.
15 Oct 2009 14:31 Default parameters are not kept in daq panel This is a known issue and will not go away soon. Shifters should learn to use the settings in the whiteboard until further notice.
15 Oct 2009 14:35 How to get Trigger Configuration of current run? The trigger configuration files are set in the "trigger" tab of the run control gui. But these file names can be very cryptic. So the most efficient way to get the trigger configuration is by asking the trigger shifter directly.
15 Oct 2009 14:37 How do I easily get to the DQ Flag upload page? There is an automatic link in the "TRT" menu. ID->TRT->Monitoring->DQ FLag Upload
This will pop up a browser directly to the web page. The username and password should be written in the HardCopy instructions at the TRT Desk.
15 Oct 2009 14:39 How does one insert a DCS alarm into the elog? Right clicking on the Warning item in the warning panel will bring up a menu list. One of the menu items is "insert to elog". Click on this item.
15 Oct 2009 14:41 How does one tell when something has been disabled from the readout? The first clue to this will be messages in the MRS. The next way to check this is by looking at the Cobra panel. Items that have been automatically removed will be marked as "MASKED".
15 Oct 2009 14:42 What is the difference between the Log Service Manager and the MRS? The MRS presents messages from the current run. The Log Service Manager can present messages from previous runs.
15 Oct 2009 14:44 The Control room machine for the TRT crashes when trying to open PPT files. (Open Office crashes). This is a new bug and will be followed up by Jim D.

TRT Expert Monitoring

Connecting to the Point 1 network

To connect to Point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh Here you are prompted for a hostname, the TRT Athena monitoring machine is pc-tdq-mon-74.

General structure of the on-line monitoring

When the jobOptions files that run the monitoring processes that appear in the DAQ Gui are changed, the oks system does not need to be run agian, the process just needs to be restarted by the run controller. (The OKS only needs to be rerun when there are changes in the xml files)

ATLAS software at Point 1

The ATLAS software is located in the directory /sw/atlas. There you can find the latestest releases that are installed at Point 1

Database access at Point 1

The TRT Monitoring segment for the combined run

The segment for the combined run is located in the OKS subdirectory /atlas/oks/tdaq-04-00-01/trt/segments/monitoring.

Log files

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) on the computer where the process is running. (Where is this configured???) To determine the computer on which the process is running
  • When running with ATLAS partition the log files are stored in (/local_sw/logs/tdaq-04-00-01/ATLAS)

SFIs and SFOs

The L2 decision is made after we get the events from the emon sampler. The online monitoring gets events from the SFI (Sub Farm Input) which basically assembles the full events and then passes them on. the L2 decision is made in SFO's (sub farm output) which are different machines than SFI's.

Running the spy GUI

For the Spy GUI, you can do igui_start. Usually you will get it in spy mode, but it can happen that you'll get expert rights. If that happens, go to the "Access Control" menu and choose "Status Display" instead of "Expert Control".


source the DAQ: source /det/tdaq/scripts/

then start the GUI: setup_daq -p ATLAS -d /atlas/oks/tdaq-04-00-01/combined/partitions/

Running the DAQ panel

The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. When you open it, you'll need to enter the proper setup script and partition file:

Setup Script: /det/tdaq/scripts/ Part Name: ATLAS Database File: /atlas/oks/tdaq-04-00-01/combined/partitions/

Stand-Alone Partition

Running a stand alone partition is a good way to debug the monitoring framework with the online system. The following describes how to create your own stand along partion and how to run it.

Creating A Stand Alone Partition

Running A Stand Alone Partition

The current working stand alone partition for TRT Monitoring is located at /det/trt/TRTMonitoring/test_partition/tdaq-04-00-01/ Go to that directory and start the partition by running ./start_test_partition.

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)

Tips, hints and fun facts

  • In order to match a L1 item (eg: L1_EM3) to the bit (eg: decimal number) go to the run list, select a run, and go to the trigger details tag.
  • To edit this page from outside Point 1 you have to go through the proxy setup. Go here and click on proxy setup.

TRT (and ATLAS general) webscripts at Point 1


A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point 1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).

Location and access

Look at the SysAdmins/WebServers TWiki to learn more about the setup of webservers and web directories at Point 1. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from in order to be able to modify or add scripts.

Scripts can be executed in a webbrowser from inside Point 1 via or from outside Point 1 through


It is important to adhere to P1 security requirements, a few guidelines are:

  1. Keep the number of scripts small, i.e. avoid ending up with a cluttered area
  2. Keep the number of persons small that have access to this area (ideally one per subsystem)
  3. Follow secure scripting principles to avoid hacker attacks through code injection/cross-side-scripting (TDAQ sysadmins and CERN IT Security will regularly test scripts for vulnerabilities):
    • Avoid filesystem and "power" commands like eval(), exec() etc., if possible also fopen
    • In case you have to use text input fields make sure to disallow any tags or even all non-alphanumerical characters as input (use re.sub('[^0-9a-zA-Z]','',<string>))
  4. Do not store unused or old scripts in the cgi-executable directory. You can temporarily rename them to .py.old or .php.old (web access to .old files is denied). Ideally move them outside the webscripts area.
  5. If you are unsure about your script security do not hesitate to contact
  6. Please make sure that the script has your name/email and ATLAS detector/group in the comments at the beginning, such that it will easier for the ATLAS TDAQ sysadmins to contact the responsible person in case of problems.

How to include scripts in a TWiki page

In order to include the scripts on a TWiki page one has to use so-called "inline frames". The <iframe> tag defines an inline frame that contains another document. It is supported in all major browsers. More documentation can be found at Example:

    <iframe width="600px" height="200px" scrolling="no" frameborder="0" src="/webscripts/global/">This text is displayed if a browser does not recognize the iframe tag.</iframe>
Note you should not include a hostname in the link, so the script is requested from the same host as the TWiki page itself (i. e. pc-atlas-www internally, atlasop externally).

List of scripts

Script name Used by Description TRT The script queries the dates of various TRT calibration scans from an Oracle database and displays them in a calendar. The next foreseen scan dates are indicated. In the widget mode a small box shows the outstanding scans to the shifter or displays OK if all scans are still valid.
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem Whiteboard.
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem. various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.
Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r19 - 2011-02-04 - vogel
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ATLAS?Please contact the page author (see Topic revision above) or the Run Coordinator of the specific system.
Contact SysAdmins support only for technical issues