Difference: TRTDetectorOperationManualExpert (1 vs. 21)

Revision 212012-02-01 - vogel

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 19 to 19
 

<-- /editTable -->
Changed:
<
<
15 Oct 2009 What to do if there are no default parameters in the DAQ Panel? The recommended values can always be found on the TRT Whiteboard.
>
>
15 Oct 2009 What to do if there are no default parameters in the DAQ Panel? The recommended values are given on the TRT Whiteboard.
 
15 Oct 2009 How to get the trigger configuration of the current run? The easiest way is to ask the trigger shifter.
Changed:
<
<
15 Oct 2009 How does one insert a DCS alarm into the elog? Right-clicking on the alarm will bring up a menu. One of the menu items is "insert to elog".
>
>
15 Oct 2009 How to insert a DCS alarm into the e-log? Right-click on the alarm, select "insert to elog" from the context menu.
 
15 Oct 2009 How to tell if some parts of the TRT have been disabled from the readout? Look at DQMD (areas flagged black), the Cobra Panel (items marked as masked), and MRS messages.
Changed:
<
<
15 Oct 2009 What is the difference between the MRS Monitor and the Log Manager? The MRS Monitor presents only current messages, the Log Manager can browse through the entire archive of MRS messages from the past.
>
>
15 Oct 2009 What is the difference between the MRS Monitor and the Log Manager? The MRS Monitor presents only new messages, the Log Manager can browse through the entire archive of past MRS messages.
 

TRT Expert Monitoring

Line: 43 to 43
 

Log Files

Changed:
<
<
  • When running the standalone partition the log files are stored in /tmp/[PartitionName] on the computer where the process is running. The host on which the process is running is shown in the “Advanced” tab of the IGUI.
  • When running with the ATLAS partition the log files are stored in /local_sw/logs/tdaq-04-00-01/ATLAS. They also appear in the partlogs web interface.
>
>
The log files of all applications are stored in /logs/tdaq-04-00-01/ on the host where each process is running. The host is shown in the “Advanced” tab of the IGUI. Log files can also be accessed through the partlogs web interface.
 

Running the IGUI

Line: 60 to 59
 

Creating a Stand-Alone Partition

Changed:
<
<
  • set up the TDAQ environment by sourcing /det/tdaq/scripts/setup_TDAQ.sh
  • set up a recent Athena version using AtlasSetup
  • get a raw data file that will serve as input for the test partition
  • run build_partition.sh with that file to create a skeleton partition
  • add the missing pieces (monitoring segment, DQM segment) to the partition
>
>
  • Set up the TDAQ environment by sourcing /det/tdaq/scripts/setup_TDAQ.sh.
  • Set up a recent Athena version using AtlasSetup, which is available under /sw/atlas/AtlasSetup.
  • Get a raw data file that will serve as input for the test partition. You can use copySFO (“Copy File”) to get a get a raw data file from an SFO.
  • Run build_partition.sh with that file to create a skeleton partition.
  • Add the missing pieces (DQM segment with DQ Agent, monitoring segment with Athena) to the partition manually.
 

Running a Stand-Alone Partition

Line: 111 to 110
 
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem.
lastruns.py various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.
Changed:
<
<
META PREFERENCE name="TDAQ_VERSION" title="TDAQ_VERSION" type="Set" value="tdaq-03-00-01"
>
>
META PREFERENCE name="TDAQ_VERSION" title="TDAQ_VERSION" type="Set" value="tdaq-04-00-01"

Revision 202011-07-14 - vogel

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Changed:
<
<
>
>
 

TRT Shift Trainer Help

Added:
>
>
 This area is intended to explain the shift Trainer's duties and to answer frequently asked questions.
Changed:
<
<

TRT Shift Trainer's Duties

The TRT TTP are the TRT shift Training Personnel. There is a week long shift equivalent to 1 shift credit (if you think this needs to be increased, feel free to ask me to increase the credit). Most of the work currently is for supervising the practical sessions. To sign up, use OTP task 529095.
  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 2pm to 5pm in the ID satellite control room. Trainee shifters sign up to OTP task ID 30910. There are only 4 slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
  • Answer questions that the trainees have. New shifters may ask questions on the email list (atlas-trt-ttp@cern.ch) or at the practical sessions. Answer the questions to the best of your ability. If you encounter a question (or suggestion) that you cannot answer please post it the FAQs below and/or send an email to the ttp email list.
>
>

TRT Shift Trainer Duties

The TRT TTP are the TRT shift Training Personnel. There is a week-long shift equivalent to one shift credit (if you think this needs to be increased, feel free to ask). Most of the work currently is for supervising the practical sessions. To sign up, use OTP task 529095.

  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 14:00 to 17:00 in the ID Satellite Control Room (building 3159). Trainee shifters sign up to OTP task ID 30910. There are only four slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
  • Answer questions that the trainees have. New shifters may ask questions on the email list (atlas-trt-ttp@cern.ch) or at the practical sessions. Answer the questions to the best of your ability. If you encounter a question (or suggestion) that you cannot answer please post it the FAQs below and/or send an email to the TTP email list.
 
  • To try to schedule special sessions for shifters who cannot attend a regular session before their first shift. The extra sessions should be scheduled with the TTP who is on shift during the week if possible. If the on shift TTP cannot schedule a special session then they need to inform the other TTPs by emailing the TTP list and asking for a volunteer.
Changed:
<
<
  • Make sure that documentation is up-to date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available here NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
  • THE FOLLOWING IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
>
>
  • Make sure that documentation is up-to date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available at NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
 
  • Brand new shifters may also require an orientation session. This should be scheduled between the TTP on shift and the brand new shifter.
Deleted:
<
<

TRT Shift Trainer's FAQ's

 
Changed:
<
<
<-- /editTable -->
>
>

TRT Shift Trainer FAQs

<-- /editTable -->
 
Date Question Answer
Changed:
<
<
15 Oct 2009 14:29 Where to find checklists filled out by previous shifters? There is a bug in the checklists mechanism that uploads the checklists to the elog. This is to be Followed up by Jim D. Usually once checklists are filled out they should go to the elog automatically.
15 Oct 2009 14:31 Default parameters are not kept in daq panel This is a known issue and will not go away soon. Shifters should learn to use the settings in the whiteboard until further notice.
15 Oct 2009 14:35 How to get Trigger Configuration of current run? The trigger configuration files are set in the "trigger" tab of the run control gui. But these file names can be very cryptic. So the most efficient way to get the trigger configuration is by asking the trigger shifter directly.
15 Oct 2009 14:37 How do I easily get to the DQ Flag upload page? There is an automatic link in the "TRT" menu. ID->TRT->Monitoring->DQ FLag Upload
This will pop up a browser directly to the web page. The username and password should be written in the HardCopy instructions at the TRT Desk.
15 Oct 2009 14:39 How does one insert a DCS alarm into the elog? Right clicking on the Warning item in the warning panel will bring up a menu list. One of the menu items is "insert to elog". Click on this item.
15 Oct 2009 14:41 How does one tell when something has been disabled from the readout? The first clue to this will be messages in the MRS. The next way to check this is by looking at the Cobra panel. Items that have been automatically removed will be marked as "MASKED".
15 Oct 2009 14:42 What is the difference between the Log Service Manager and the MRS? The MRS presents messages from the current run. The Log Service Manager can present messages from previous runs.
15 Oct 2009 14:44 The Control room machine for the TRT crashes when trying to open PPT files. (Open Office crashes). This is a new bug and will be followed up by Jim D.
>
>
15 Oct 2009 What to do if there are no default parameters in the DAQ Panel? The recommended values can always be found on the TRT Whiteboard.
15 Oct 2009 How to get the trigger configuration of the current run? The easiest way is to ask the trigger shifter.
15 Oct 2009 How does one insert a DCS alarm into the elog? Right-clicking on the alarm will bring up a menu. One of the menu items is "insert to elog".
15 Oct 2009 How to tell if some parts of the TRT have been disabled from the readout? Look at DQMD (areas flagged black), the Cobra Panel (items marked as masked), and MRS messages.
15 Oct 2009 What is the difference between the MRS Monitor and the Log Manager? The MRS Monitor presents only current messages, the Log Manager can browse through the entire archive of MRS messages from the past.
 

TRT Expert Monitoring

Changed:
<
<

Connecting to the Point 1 network

To connect to Point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh atlasgw.cern.ch. Here you are prompted for a hostname, the TRT Athena monitoring machine is pc-tdq-mon-74.
>
>

Connecting to the Point-1 Network

 
Changed:
<
<

General structure of the on-line monitoring

When the jobOptions files that run the monitoring processes that appear in the DAQ Gui are changed, the oks system does not need to be run agian, the process just needs to be restarted by the run controller. (The OKS only needs to be rerun when there are changes in the xml files)
>
>
See the Sysadmins FAQ, section “Gateways”. In short, connect to atlasgw.cern.ch and then hop to your destination host.
 
Changed:
<
<

ATLAS software at Point 1

The ATLAS software is located in the directory /sw/atlas. There you can find the latestest releases that are installed at Point 1
>
>

ATLAS Software at Point 1

 
Changed:
<
<

Database access at Point 1

>
>
The ATLAS software is located in the directory /sw/atlas. There you can find the latestest releases that are installed at Point 1
 
Changed:
<
<

The TRT Monitoring segment for the combined run

The segment for the combined run is located in the OKS subdirectory /atlas/oks/tdaq-04-00-01/trt/segments/monitoring.
>
>

Database Access at Point 1

 
Changed:
<
<

Log files

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) on the computer where the process is running. (Where is this configured???) To determine the computer on which the process is running
  • When running with ATLAS partition the log files are stored in (/local_sw/logs/tdaq-04-00-01/ATLAS)
>
>

The TRT Monitoring Segment for the Combined Run

 
Changed:
<
<

SFIs and SFOs

The L2 decision is made after we get the events from the emon sampler. The online monitoring gets events from the SFI (Sub Farm Input) which basically assembles the full events and then passes them on. the L2 decision is made in SFO's (sub farm output) which are different machines than SFI's.
>
>
The segment for the combined run is located in the OKS subdirectory /atlas/oks/tdaq-04-00-01/trt/segments/monitoring.
 
Changed:
<
<

Running the spy GUI

For the Spy GUI, you can do igui_start. Usually you will get it in spy mode, but it can happen that you'll get expert rights. If that happens, go to the "Access Control" menu and choose "Status Display" instead of "Expert Control".
>
>

Log Files

 
Changed:
<
<
Alternatively:
>
>
  • When running the standalone partition the log files are stored in /tmp/[PartitionName] on the computer where the process is running. The host on which the process is running is shown in the “Advanced” tab of the IGUI.
  • When running with the ATLAS partition the log files are stored in /local_sw/logs/tdaq-04-00-01/ATLAS. They also appear in the partlogs web interface.
 
Changed:
<
<
source the DAQ: source /det/tdaq/scripts/setup_TDAQ.sh
>
>

Running the IGUI

 
Changed:
<
<
then start the GUI: setup_daq -p ATLAS -d /atlas/oks/tdaq-04-00-01/combined/partitions/ATLAS.data.xml
>
>
First source the DAQ environment with source /det/tdaq/scripts/setup_TDAQ.sh. Then you can start the IGUI with igui_start -p or alternatively setup_daq -p ATLAS -d /atlas/oks/tdaq-04-00-01/combined/partitions/ATLAS.data.xml.
 
Changed:
<
<

Running the DAQ panel

The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. When you open it, you'll need to enter the proper setup script and partition file:
>
>

Running the DAQ Panel

 
Changed:
<
<
Setup Script: /det/tdaq/scripts/setup_TDAQ.sh Part Name: ATLAS Database File: /atlas/oks/tdaq-04-00-01/combined/partitions/ATLAS.data.xml
>
>
The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. When you open it, you'll need to enter some settings that are provided on the TRT Whiteboard.
 

Stand-Alone Partition

Deleted:
<
<
Running a stand alone partition is a good way to debug the monitoring framework with the online system. The following describes how to create your own stand along partion and how to run it.
 
Changed:
<
<

Creating A Stand Alone Partition

>
>
Running a stand-alone partition is a good way to debug the monitoring framework with the online system. The following describes how to create your own stand along partion and how to run it.
 
Changed:
<
<

Running A Stand Alone Partition

>
>

Creating a Stand-Alone Partition

 
Changed:
<
<
The current working stand alone partition for TRT Monitoring is located at /det/trt/TRTMonitoring/test_partition/tdaq-04-00-01/TRTMonitoringTest.data.xml. Go to that directory and start the partition by running ./start_test_partition.
>
>
  • set up the TDAQ environment by sourcing /det/tdaq/scripts/setup_TDAQ.sh
  • set up a recent Athena version using AtlasSetup
  • get a raw data file that will serve as input for the test partition
  • run build_partition.sh with that file to create a skeleton partition
  • add the missing pieces (monitoring segment, DQM segment) to the partition
 
Changed:
<
<

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)
>
>

Running a Stand-Alone Partition

 
Changed:
<
<

Tips, hints and fun facts

  • In order to match a L1 item (eg: L1_EM3) to the bit (eg: decimal number) go to the run list, select a run, and go to the trigger details tag.
  • To edit this page from outside Point 1 you have to go through the proxy setup. Go here and click on proxy setup.
>
>
The current working stand alone partition for TRT Monitoring is located at /det/trt/monitoring/test_partition/tdaq-04-00-01/TRTMonitoringTest.data.xml. Go to that directory and start the partition by running ./start_test_partition.
 
Changed:
<
<

TRT (and ATLAS general) webscripts at Point 1

>
>

Webscripts at Point 1

 

Purpose

Deleted:
<
<
A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point 1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).
 
Changed:
<
<

Location and access

>
>
A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point 1 TWiki pages e.g. in an iframe as part of the Whiteboard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).
 
Changed:
<
<
Look at the SysAdmins/WebServers TWiki to learn more about the setup of webservers and web directories at Point 1. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from atlas-tdaq.sysadmin@cern.ch in order to be able to modify or add scripts.
>
>

Location and Access

Look at the SysAdmins/WebServers TWiki to learn more about the setup of webservers and web directories at Point 1. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from atlas-tdaq-sysadmins@cern.ch to access the web server.

  Scripts can be executed in a webbrowser from inside Point 1 via http://pc-atlas-www.cern.ch/webscripts/... or from outside Point 1 through https://atlasop.cern.ch/webscripts/....

Guidelines

It is important to adhere to P1 security requirements, a few guidelines are:

Changed:
<
<
  1. Keep the number of scripts small, i.e. avoid ending up with a cluttered area
  2. Keep the number of persons small that have access to this area (ideally one per subsystem)
  3. Follow secure scripting principles to avoid hacker attacks through code injection/cross-side-scripting (TDAQ sysadmins and CERN IT Security will regularly test scripts for vulnerabilities):
    • Avoid filesystem and "power" commands like eval(), exec() etc., if possible also fopen
>
>
  1. Keep the number of scripts small, i.e. avoid ending up with a cluttered area
  2. Keep the number of persons small that have access to this area (ideally one per subsystem)
  3. Follow secure scripting principles to avoid hacker attacks through code injection/cross-side-scripting (TDAQ sysadmins and CERN IT Security will regularly test scripts for vulnerabilities):
    • Avoid filesystem and "power" commands like eval(), exec() etc., if possible also fopen()
 
    • In case you have to use text input fields make sure to disallow any tags or even all non-alphanumerical characters as input (use re.sub('[^0-9a-zA-Z]','',<string>))
Changed:
<
<
  1. Do not store unused or old scripts in the cgi-executable directory. You can temporarily rename them to .py.old or .php.old (web access to .old files is denied). Ideally move them outside the webscripts area.
  2. If you are unsure about your script security do not hesitate to contact atlas-tdaq.sysadmin@cern.ch.
  3. Please make sure that the script has your name/email and ATLAS detector/group in the comments at the beginning, such that it will easier for the ATLAS TDAQ sysadmins to contact the responsible person in case of problems.
>
>
  1. Do not store unused or old scripts in the cgi-executable directory. You can temporarily rename them to .py.old or .php.old (web access to .old files is denied). Ideally move them outside the webscripts area.
  2. If you are unsure about your script security do not hesitate to contact atlas-tdaq-sysadmins@cern.ch.
  3. Please make sure that the script has your name/email and ATLAS detector/group in the comments at the beginning, such that it will easier for the ATLAS TDAQ sysadmins to contact the responsible person in case of problems.
 
Changed:
<
<

How to include scripts in a TWiki page

>
>

How to Include Scripts in a TWiki Page

 
Changed:
<
<
In order to include the scripts on a TWiki page one has to use so-called "inline frames". The <iframe> tag defines an inline frame that contains another document. It is supported in all major browsers. More documentation can be found at http://www.w3.org/TR/REC-html40/present/frames.html#edef-IFRAME.
>
>
In order to include the scripts on a TWiki page you can use "inline frames". The <iframe> tag defines an inline frame that contains another document. It is supported in all major browsers. More documentation can be found at W3C.
 Example:
    <iframe width="600px" height="200px" scrolling="no" frameborder="0" src="/webscripts/global/lastruns.py">This text is displayed if a browser does not recognize the iframe tag.</iframe>
Note you should not include a hostname in the link, so the script is requested from the same host as the TWiki page itself (i. e. pc-atlas-www internally, atlasop externally).
Changed:
<
<

List of scripts

>
>

List of Scripts

 
Script name Used by Description
calibcal.py TRT The script queries the dates of various TRT calibration scans from an Oracle database and displays them in a calendar. The next foreseen scan dates are indicated. In the widget mode a small box shows the outstanding scans to the shifter or displays OK if all scans are still valid.

Revision 192011-02-04 - vogel

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 120 to 120
 
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem Whiteboard.
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem.
lastruns.py various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.
Changed:
<
<
META PREFERENCE name="TDAQ_VERSION" title="TDAQ_VERSION" type="Set" value="03-00-01"
>
>
META PREFERENCE name="TDAQ_VERSION" title="TDAQ_VERSION" type="Set" value="tdaq-03-00-01"

Revision 182011-02-04 - vogel

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 120 to 120
 
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem Whiteboard.
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem.
lastruns.py various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.
Added:
>
>
META PREFERENCE name="TDAQ_VERSION" title="TDAQ_VERSION" type="Set" value="03-00-01"

Revision 172011-02-04 - vogel

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 13 to 13
 
  • THE FOLLOWING IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
  • Brand new shifters may also require an orientation session. This should be scheduled between the TTP on shift and the brand new shifter.

TRT Shift Trainer's FAQ's

Changed:
<
<
>
>
 
Date Question Answer
15 Oct 2009 14:29 Where to find checklists filled out by previous shifters? There is a bug in the checklists mechanism that uploads the checklists to the elog. This is to be Followed up by Jim D. Usually once checklists are filled out they should go to the elog automatically.
<-- /editTable -->
Line: 24 to 24
 
15 Oct 2009 14:41 How does one tell when something has been disabled from the readout? The first clue to this will be messages in the MRS. The next way to check this is by looking at the Cobra panel. Items that have been automatically removed will be marked as "MASKED".
15 Oct 2009 14:42 What is the difference between the Log Service Manager and the MRS? The MRS presents messages from the current run. The Log Service Manager can present messages from previous runs.
15 Oct 2009 14:44 The Control room machine for the TRT crashes when trying to open PPT files. (Open Office crashes). This is a new bug and will be followed up by Jim D.
Deleted:
<
<
 

TRT Expert Monitoring

Changed:
<
<

Connecting to the Point 1 network

To connect to point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh atlasgw.cern.ch. Here you are prompted for a hostname, the two TRT Athena monitoring machines are pc-tdq-mon-24 and pc-tdq-mon-25
>
>

Connecting to the Point 1 network

To connect to Point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh atlasgw.cern.ch. Here you are prompted for a hostname, the TRT Athena monitoring machine is pc-tdq-mon-74.
 

General structure of the on-line monitoring

When the jobOptions files that run the monitoring processes that appear in the DAQ Gui are changed, the oks system does not need to be run agian, the process just needs to be restarted by the run controller. (The OKS only needs to be rerun when there are changes in the xml files)
Changed:
<
<

Atlas software at Point 1

The atlas software is located in the directory /sw/atlas/. There you can find the latestest releases that are installed at point 1
>
>

ATLAS software at Point 1

The ATLAS software is located in the directory /sw/atlas. There you can find the latestest releases that are installed at Point 1
 
Changed:
<
<

Database access at Point 1

>
>

Database access at Point 1

 

The TRT Monitoring segment for the combined run

Changed:
<
<
The segment for the combined run is located in /db/tdaq-01-09-01/trt/segments/Monitoring/ (where 01-09-01 is the latest version of the tdaq)
>
>
The segment for the combined run is located in the OKS subdirectory /atlas/oks/tdaq-04-00-01/trt/segments/monitoring.
 

Log files

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) on the computer where the process is running. (Where is this configured???) To determine the computer on which the process is running
Changed:
<
<
  • When running with Atlas partition the log files are stored in (/logs/tdaq-01-09-01/ATLAS)
>
>
  • When running with ATLAS partition the log files are stored in (/local_sw/logs/tdaq-04-00-01/ATLAS)
 
Changed:
<
<

SFI's and all that jazz

>
>

SFIs and SFOs

 The L2 decision is made after we get the events from the emon sampler. The online monitoring gets events from the SFI (Sub Farm Input) which basically assembles the full events and then passes them on. the L2 decision is made in SFO's (sub farm output) which are different machines than SFI's.

Running the spy GUI

Line: 57 to 56
  source the DAQ: source /det/tdaq/scripts/setup_TDAQ.sh
Changed:
<
<
then start the GUI: setup_daq -p ATLAS -d /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml
>
>
then start the GUI: setup_daq -p ATLAS -d /atlas/oks/tdaq-04-00-01/combined/partitions/ATLAS.data.xml
 

Running the DAQ panel

Changed:
<
<
The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. (And I think it might even be different between the SCR machines and the atlas-cr machines.) When you open it, you'll need to enter the proper setup script and partition file:
>
>
The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. When you open it, you'll need to enter the proper setup script and partition file:
 
Changed:
<
<
Setup Script: /det/tdaq/scripts/setup_TDAQ_14.2.XY.sh Part Name: ATLAS Database File: /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml
>
>
Setup Script: /det/tdaq/scripts/setup_TDAQ.sh Part Name: ATLAS Database File: /atlas/oks/tdaq-04-00-01/combined/partitions/ATLAS.data.xml
 

Stand-Alone Partition

Running a stand alone partition is a good way to debug the monitoring framework with the online system. The following describes how to create your own stand along partion and how to run it.
Line: 73 to 72
 

Running A Stand Alone Partition

Changed:
<
<
The current working stand alone partition for TRT Monitoring is located at
 /det/trt/TRTMonitoring/R14.5.0_tdaq-2.0_debug/WorkDir/run/part_jdegenha.data.xml 
cd to that directory and then source the SetupDAQCOmmissioning.sh script.
$cd /det/trt/TRTMonitoring/R14.5.0_tdaq-2.0_debug/WorkDir/run/
$source SetupDAQCommissioning.sh
Then start up the partition.
$setup_daq -p part_jdegenha -d part_jdegenha.data.xml 

Running Athena Stateless Monitoring

Currently the Athena Stateless Monitoring cannot run inside the DAQ partition due to database issues. The next best thing is to run Athena Stateless monitoring separately and have it connect to a running partition. To do this, first get a running partition as stated above. Next, in a new terminal session, cd to
/det/trt/TRTMonitoring/Official_JOs 

Then source the config.sh script, and then run athena:

$source config.sh
$athena AthenaMonitoringStateless_JobOptions.py

Then check in the running partition if there are TRT histograms in the OH.

Installing a new version of the TRT monitoring

Running TRT Monitoring online is simply a matter of using the right job options file. The job options file should be found in /sw/atlas/AtlasOffline//.

1. create a TESTAREA under /det/trt/TRTMonitoring/. 2. in the TESTAREA create a cmthome directory, and WorkDir:

 
mkdir /det/trt/TRTMonitoring/<TESTAREA>/cmthome
mkdir /det/trt/TRTMonitoring/<TESTAREA>/WorkDir
3. in this cmthome area create requirements file:

set CMTSITE STANDALONE
set SITEROOT /sw/atlas
macro ATLAS_DIST_AREA ${SITEROOT}
macro HLT_PROJECT_AREA ${SITEROOT}
macro ATLAS_TEST_AREA /det/trt/TRTMonitoring/<TESTAREA>/WorkDir
macro ATLAS_GROUP_AREA /sw/atlas/<PROJECT>/<RELEASE>
apply_tag groupArea
apply_tag oneGroup
apply_tag oneTest
apply_tag projectArea
macro SITE_PROJECT_AREA ${SITEROOT}
macro EXTERNAL_PROJECT_AREA ${SITEROOT}
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
set CMTCONFIG i686-slc4-gcc34-opt

then

source /sw/atlas/CMT/v1r20p20080222/mgr/setup.sh 
cmt config

Log out of that session and then log back in.

 
cd /det/trt/TRTMonitoring/<TESTAREA>/
mkdir 
source cmthome/setup.sh -tag=<RELEASE>,setup,32,runtime,<PROJECT>
export CORAL_AUTH_PATH=/sw/DbSuppForMx
export CORAL_DBLOOKUP_PATH=/sw/DbSuppForMx
cd WorkDir
mkdir run
cd run
cp /sw/atlas/AtlasTier0/<RELEASE>/InnerDetector/InDetExample/InDetRecExample/share/jobOptions_cosmic.py .

Edit

 jobOptions_cosmic.py 
appropriately to get the monitoring to run.

athena jobOptions_cosmic.py |tee monitoring.log

You should get a root file called

monitoring.root
. Check to make sure that the file has correctly filled histograms.

This is the first step in getting Athena monitoring to run at point1.

>
>
The current working stand alone partition for TRT Monitoring is located at /det/trt/TRTMonitoring/test_partition/tdaq-04-00-01/TRTMonitoringTest.data.xml. Go to that directory and start the partition by running ./start_test_partition.
 

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)
Deleted:
<
<

Integrating in a new run CASE STUDY (cosmic run 2/10/8)

The monitoring runs ok, out of the box, however some of SFIs are not seeing events from the gather. Problem is that the L1 triggers were configured incorrectly. Need to locate a script (from rhauser) which gives you the L1 bit given the L1 type to configure the monitoring correctly. The script is /atlas-home/1/rhauser/bin/l1bits4emon. The L1 bits are set in the top jobOptions for each monitoring processes, which configure the monitoring process by providing the L1 bit and the

Trigger Presenter

Displays the trigger chains for the triggers in the partitions, which L1s feed into which L1 ect. The Trigger Presenter is avialible through the TDAQ panel.
 

Tips, hints and fun facts

  • In order to match a L1 item (eg: L1_EM3) to the bit (eg: decimal number) go to the run list, select a run, and go to the trigger details tag.
Changed:
<
<
  • To copy files from lxplus ...
  • To edit this page from outside point 1 you have to go through the proxy setup. Go here and click on proxy setup.
>
>
  • To edit this page from outside Point 1 you have to go through the proxy setup. Go here and click on proxy setup.
 
Changed:
<
<

TRT (and ATLAS general) webscripts at Point1

>
>

TRT (and ATLAS general) webscripts at Point 1

 

Purpose

Changed:
<
<
A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).
>
>
A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point 1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).
 

Location and access

Changed:
<
<
Physically the webscripts are located on pc-atlas-www-1. A specific folder has been assigned to accommodate general as well subsystem-specific scripts. Directories are cgi-executable and set up to give visibility to users connecting from outside Point1 via atlasop.cern.ch. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from atlas-tdaq.sysadmin@cern.ch in order to be able to modify or add scripts.

P1 machine pc-atlas-www-1.cern.ch
scripts root folder /www/web_files/html/webscripts/
subsystem subdirs /www/web_files/html/webscripts/trt/
/www/web_files/html/webscripts/tile/
...

Scripts can be executed in a webbrowser from inside Point1 via
https://pc-atlas-www.cern.ch/webscripts/...
or from outside Point1 through
https://atlasop.cern.ch/webscripts/....

>
>
Look at the SysAdmins/WebServers TWiki to learn more about the setup of webservers and web directories at Point 1. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from atlas-tdaq.sysadmin@cern.ch in order to be able to modify or add scripts.
 
Added:
>
>
Scripts can be executed in a webbrowser from inside Point 1 via http://pc-atlas-www.cern.ch/webscripts/... or from outside Point 1 through https://atlasop.cern.ch/webscripts/....
 
Changed:
<
<

Guidelines (IMPORTANT)

It is IMPORTANT to adhere to P1 security requirements, a few guidelines are:

>
>

Guidelines

 
Added:
>
>
It is important to adhere to P1 security requirements, a few guidelines are:
 
  1. Keep the number of scripts small, i.e. avoid ending up with a cluttered area
  2. Keep the number of persons small that have access to this area (ideally one per subsystem)
  3. Follow secure scripting principles to avoid hacker attacks through code injection/cross-side-scripting (TDAQ sysadmins and CERN IT Security will regularly test scripts for vulnerabilities):
Line: 201 to 109
 In order to include the scripts on a TWiki page one has to use so-called "inline frames". The <iframe> tag defines an inline frame that contains another document. It is supported in all major browsers. More documentation can be found at http://www.w3.org/TR/REC-html40/present/frames.html#edef-IFRAME. Example:
Changed:
<
<
... <iframe width="600px" height="200px" scrolling="no" frameborder="0" src="http://pc-atlas-www.cern.ch/webscripts/lastruns.py">This text is displayed if a browser does not recognize the iframe tag. ...
>
>
 
Changed:
<
<
It is necessary to use TWiki variables so that the script is executed on the same server than the TWiki page (i.e. atlasop.cern.ch resp. pc-atlas-www-1.cern.ch). TWiki variable are set in the TWiki page settings:
  1. Goto the bottom of the TWiki page and click on More topic actions.
  2. Under "Edit topic preference settings" click "Edit settings for this topic".
  3. Add variable(s) as described there.
  4. Example:
    * Set MYPATH = %IF{"$ HTTP_HOST = 'pc-atlas-www.cern.ch'" then="https://pc-atlas-www.cern.ch/webscripts" else="https://atlasop.cern.ch/atlas-point1/webscripts"}%
>
>
Note you should not include a hostname in the link, so the script is requested from the same host as the TWiki page itself (i. e. pc-atlas-www internally, atlasop externally).
 

List of scripts

Script name Used by Description
Changed:
<
<
calibcal.py, calibcal_slc4.py TRT The script queries the dates of various TRT calibration scans from an Oracle database and displays them in a calendar. The next foreseen scan dates are indicated. In the widget mode a small box shows the outstanding scans to the shifter or displays OK if all scans are still valid. Two versions are maintained accounting for the fact that pc-atlas-www-1 still operates under SLC4.
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem WhiteBoard.
>
>
calibcal.py TRT The script queries the dates of various TRT calibration scans from an Oracle database and displays them in a calendar. The next foreseen scan dates are indicated. In the widget mode a small box shows the outstanding scans to the shifter or displays OK if all scans are still valid.
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem Whiteboard.
 
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem.
lastruns.py various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.

Revision 162010-04-26 - pcwetans

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 164 to 164
 
  • To copy files from lxplus ...
  • To edit this page from outside point 1 you have to go through the proxy setup. Go here and click on proxy setup.
Changed:
<
<

TRT (and ATLAS general) scripts at Point1

>
>

TRT (and ATLAS general) webscripts at Point1

 

Purpose

A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).
Line: 175 to 175
 
P1 machine pc-atlas-www-1.cern.ch
scripts root folder /www/web_files/html/webscripts/
Changed:
<
<
subsystem subdirs /www/web_files/html/webscripts/trt/
/www/web_files/html/webscripts/sct/
...
>
>
subsystem subdirs /www/web_files/html/webscripts/trt/
/www/web_files/html/webscripts/tile/
...
  Scripts can be executed in a webbrowser from inside Point1 via
https://pc-atlas-www.cern.ch/webscripts/...

Revision 152010-04-26 - pcwetans

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 163 to 163
 
  • In order to match a L1 item (eg: L1_EM3) to the bit (eg: decimal number) go to the run list, select a run, and go to the trigger details tag.
  • To copy files from lxplus ...
  • To edit this page from outside point 1 you have to go through the proxy setup. Go here and click on proxy setup.
Added:
>
>

TRT (and ATLAS general) scripts at Point1

Purpose

A number of scripts have been written to create tables or widgets with information relevant to shifters. They can be included on the Point1 TWiki pages e.g. in an iframe as part of the WhiteBoard or shifter instructions. The scripts are mostly written in Python, some might be coded in PHP. A few scripts are TRT-specific (e.g. calibcal), others can be and are used by other ATLAS subsystems (e.g. lastruns or dayprog).

Location and access

Physically the webscripts are located on pc-atlas-www-1. A specific folder has been assigned to accommodate general as well subsystem-specific scripts. Directories are cgi-executable and set up to give visibility to users connecting from outside Point1 via atlasop.cern.ch. The responsible subsystem experts have to demand access rights. In particular you need to request the role ATLAS:www from atlas-tdaq.sysadmin@cern.ch in order to be able to modify or add scripts.

P1 machine pc-atlas-www-1.cern.ch
scripts root folder /www/web_files/html/webscripts/
subsystem subdirs /www/web_files/html/webscripts/trt/
/www/web_files/html/webscripts/sct/
...

Scripts can be executed in a webbrowser from inside Point1 via
https://pc-atlas-www.cern.ch/webscripts/...
or from outside Point1 through
https://atlasop.cern.ch/webscripts/....

Guidelines (IMPORTANT)

It is IMPORTANT to adhere to P1 security requirements, a few guidelines are:

  1. Keep the number of scripts small, i.e. avoid ending up with a cluttered area
  2. Keep the number of persons small that have access to this area (ideally one per subsystem)
  3. Follow secure scripting principles to avoid hacker attacks through code injection/cross-side-scripting (TDAQ sysadmins and CERN IT Security will regularly test scripts for vulnerabilities):
    • Avoid filesystem and "power" commands like eval(), exec() etc., if possible also fopen
    • In case you have to use text input fields make sure to disallow any tags or even all non-alphanumerical characters as input (use re.sub('[^0-9a-zA-Z]','',<string>))
  4. Do not store unused or old scripts in the cgi-executable directory. You can temporarily rename them to .py.old or .php.old (web access to .old files is denied). Ideally move them outside the webscripts area.
  5. If you are unsure about your script security do not hesitate to contact atlas-tdaq.sysadmin@cern.ch.
  6. Please make sure that the script has your name/email and ATLAS detector/group in the comments at the beginning, such that it will easier for the ATLAS TDAQ sysadmins to contact the responsible person in case of problems.

How to include scripts in a TWiki page

In order to include the scripts on a TWiki page one has to use so-called "inline frames". The <iframe> tag defines an inline frame that contains another document. It is supported in all major browsers. More documentation can be found at http://www.w3.org/TR/REC-html40/present/frames.html#edef-IFRAME. Example:

...
<iframe width="600px" height="200px" scrolling="no" frameborder="0"
  src="http://pc-atlas-www.cern.ch/webscripts/lastruns.py">This text is displayed if a browser does not recognize the iframe tag.</iframe>
...

It is necessary to use TWiki variables so that the script is executed on the same server than the TWiki page (i.e. atlasop.cern.ch resp. pc-atlas-www-1.cern.ch). TWiki variable are set in the TWiki page settings:

  1. Goto the bottom of the TWiki page and click on More topic actions.
  2. Under "Edit topic preference settings" click "Edit settings for this topic".
  3. Add variable(s) as described there.
  4. Example:
    * Set MYPATH = %IF{"$ HTTP_HOST = 'pc-atlas-www.cern.ch'" then="https://pc-atlas-www.cern.ch/webscripts" else="https://atlasop.cern.ch/atlas-point1/webscripts"}%

List of scripts

Script name Used by Description
calibcal.py, calibcal_slc4.py TRT The script queries the dates of various TRT calibration scans from an Oracle database and displays them in a calendar. The next foreseen scan dates are indicated. In the widget mode a small box shows the outstanding scans to the shifter or displays OK if all scans are still valid. Two versions are maintained accounting for the fact that pc-atlas-www-1 still operates under SLC4.
dayprog.php various subsystems Displays the daily ATLAS run program in a small box, so it can be included on the subsystem WhiteBoard.
mdtdq.php, tiledq.php, trtdq.php respective subsystems Displays the DQ flag colors of the DQ regions of the respective subsystem.
lastruns.py various subsystems Returns a table of the last n runs with general run information and subsystem-specific DQ results. Direct links to the flag upload website are available.

Revision 142010-03-12 - jdegenha

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Line: 6 to 6
 This area is intended to explain the shift Trainer's duties and to answer frequently asked questions.

TRT Shift Trainer's Duties

The TRT TTP are the TRT shift Training Personnel. There is a week long shift equivalent to 1 shift credit (if you think this needs to be increased, feel free to ask me to increase the credit). Most of the work currently is for supervising the practical sessions. To sign up, use OTP task 529095.
Changed:
<
<
  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 2pm to 5pm in the ID satellite control room. Trainee shifters sign up to OTP task ID 309100. There are only 4 slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
>
>
  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 2pm to 5pm in the ID satellite control room. Trainee shifters sign up to OTP task ID 30910. There are only 4 slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
 
  • Answer questions that the trainees have. New shifters may ask questions on the email list (atlas-trt-ttp@cern.ch) or at the practical sessions. Answer the questions to the best of your ability. If you encounter a question (or suggestion) that you cannot answer please post it the FAQs below and/or send an email to the ttp email list.
Changed:
<
<
  • To try to schedule special sessions for shifter who cannot attend a regular session before their first shift. The extra sessions should be scheduled with the TTP who is on shift during the week if possible. If that on shift TTP cannot schedule a special session then they need to inform the other ttp by emailing the ttp list and asking for a volunteer.
  • Make sure that documentation is upto date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available here NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
  • THIS IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
>
>
  • To try to schedule special sessions for shifters who cannot attend a regular session before their first shift. The extra sessions should be scheduled with the TTP who is on shift during the week if possible. If the on shift TTP cannot schedule a special session then they need to inform the other TTPs by emailing the TTP list and asking for a volunteer.
  • Make sure that documentation is up-to date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available here NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
  • THE FOLLOWING IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
 
  • Brand new shifters may also require an orientation session. This should be scheduled between the TTP on shift and the brand new shifter.

TRT Shift Trainer's FAQ's

<-- /editTable -->
Changed:
<
<
Date Queston Answer
>
>
Date Question Answer
 
15 Oct 2009 14:29 Where to find checklists filled out by previous shifters? There is a bug in the checklists mechanism that uploads the checklists to the elog. This is to be Followed up by Jim D. Usually once checklists are filled out they should go to the elog automatically.
15 Oct 2009 14:31 Default parameters are not kept in daq panel This is a known issue and will not go away soon. Shifters should learn to use the settings in the whiteboard until further notice.
Changed:
<
<
15 Oct 2009 14:35 How to get Trigger Configuration of current run? The trigger configuration files are set in the "trigger" tab of the run control gui. But these file names can be very cryptic. So the most efficient way to get the trigger configuration is by asking the trigger shifter.
>
>
15 Oct 2009 14:35 How to get Trigger Configuration of current run? The trigger configuration files are set in the "trigger" tab of the run control gui. But these file names can be very cryptic. So the most efficient way to get the trigger configuration is by asking the trigger shifter directly.
 
15 Oct 2009 14:37 How do I easily get to the DQ Flag upload page? There is an automatic link in the "TRT" menu. ID->TRT->Monitoring->DQ FLag Upload
This will pop up a browser directly to the web page. The username and password should be written in the HardCopy instructions at the TRT Desk.
15 Oct 2009 14:39 How does one insert a DCS alarm into the elog? Right clicking on the Warning item in the warning panel will bring up a menu list. One of the menu items is "insert to elog". Click on this item.
15 Oct 2009 14:41 How does one tell when something has been disabled from the readout? The first clue to this will be messages in the MRS. The next way to check this is by looking at the Cobra panel. Items that have been automatically removed will be marked as "MASKED".
15 Oct 2009 14:42 What is the difference between the Log Service Manager and the MRS? The MRS presents messages from the current run. The Log Service Manager can present messages from previous runs.
15 Oct 2009 14:44 The Control room machine for the TRT crashes when trying to open PPT files. (Open Office crashes). This is a new bug and will be followed up by Jim D.
Deleted:
<
<
15 Oct 2009 14:45 Many DQM histograms are using argon as a reference! At least the file name of the reference histograms has Argon in the name. Shifters are noticing this and therefore ignoring the references. Jim D. will follow this up and try to get the DQ guys to use the right references.
 

TRT Expert Monitoring

Revision 132009-10-15 - jdegenha

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Changed:
<
<
-- JohnAlison - 02 Oct 2008
>
>

TRT Shift Trainer Help

This area is intended to explain the shift Trainer's duties and to answer frequently asked questions.

TRT Shift Trainer's Duties

The TRT TTP are the TRT shift Training Personnel. There is a week long shift equivalent to 1 shift credit (if you think this needs to be increased, feel free to ask me to increase the credit). Most of the work currently is for supervising the practical sessions. To sign up, use OTP task 529095.
  • Supervise Practical Sessions with Trainee Shifters. Sessions are currently held every Wednesday from 2pm to 5pm in the ID satellite control room. Trainee shifters sign up to OTP task ID 309100. There are only 4 slots open per session, but others may arrive without signing up. Please be flexible when this happens. Make sure everyone has a chance to try out all of the shifter tools. The goal is not to make them experts, just to make them competent shifters. Also make sure that the new shifters enter their name in the table at TrtAcrShifterCertificationRecords.
  • Answer questions that the trainees have. New shifters may ask questions on the email list (atlas-trt-ttp@cern.ch) or at the practical sessions. Answer the questions to the best of your ability. If you encounter a question (or suggestion) that you cannot answer please post it the FAQs below and/or send an email to the ttp email list.
  • To try to schedule special sessions for shifter who cannot attend a regular session before their first shift. The extra sessions should be scheduled with the TTP who is on shift during the week if possible. If that on shift TTP cannot schedule a special session then they need to inform the other ttp by emailing the ttp list and asking for a volunteer.
  • Make sure that documentation is upto date and sufficient for training new shifters. If it is not then the responsible expert needs to be informed. All of the documentation is available here NewTRTShifters, including the Shifter Training Checklist, Introductory presentations, and new shifter orientation.
  • THIS IS NOT A TASK YET In the near future, regular introductory presentations may be scheduled as well. When this is started, the ttp shift credit will increase accordingly.
  • Brand new shifters may also require an orientation session. This should be scheduled between the TTP on shift and the brand new shifter.

TRT Shift Trainer's FAQ's

Date Queston Answer
15 Oct 2009 14:29 Where to find checklists filled out by previous shifters? There is a bug in the checklists mechanism that uploads the checklists to the elog. This is to be Followed up by Jim D. Usually once checklists are filled out they should go to the elog automatically.
15 Oct 2009 14:31 Default parameters are not kept in daq panel This is a known issue and will not go away soon. Shifters should learn to use the settings in the whiteboard until further notice.
15 Oct 2009 14:35 How to get Trigger Configuration of current run? The trigger configuration files are set in the "trigger" tab of the run control gui. But these file names can be very cryptic. So the most efficient way to get the trigger configuration is by asking the trigger shifter.
15 Oct 2009 14:37 How do I easily get to the DQ Flag upload page? There is an automatic link in the "TRT" menu. ID->TRT->Monitoring->DQ FLag Upload
This will pop up a browser directly to the web page. The username and password should be written in the HardCopy instructions at the TRT Desk.
15 Oct 2009 14:39 How does one insert a DCS alarm into the elog? Right clicking on the Warning item in the warning panel will bring up a menu list. One of the menu items is "insert to elog". Click on this item.
15 Oct 2009 14:41 How does one tell when something has been disabled from the readout? The first clue to this will be messages in the MRS. The next way to check this is by looking at the Cobra panel. Items that have been automatically removed will be marked as "MASKED".
15 Oct 2009 14:42 What is the difference between the Log Service Manager and the MRS? The MRS presents messages from the current run. The Log Service Manager can present messages from previous runs.
15 Oct 2009 14:44 The Control room machine for the TRT crashes when trying to open PPT files. (Open Office crashes). This is a new bug and will be followed up by Jim D.
15 Oct 2009 14:45 Many DQM histograms are using argon as a reference! At least the file name of the reference histograms has Argon in the name. Shifters are noticing this and therefore ignoring the references. Jim D. will follow this up and try to get the DQ guys to use the right references.
<-- /editTable -->
 

TRT Expert Monitoring

Connecting to the Point 1 network

Revision 122009-03-10 - jdegenha

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 48 to 48
 

Running A Stand Alone Partition

Added:
>
>
The current working stand alone partition for TRT Monitoring is located at
 /det/trt/TRTMonitoring/R14.5.0_tdaq-2.0_debug/WorkDir/run/part_jdegenha.data.xml 
cd to that directory and then source the SetupDAQCOmmissioning.sh script.
$cd /det/trt/TRTMonitoring/R14.5.0_tdaq-2.0_debug/WorkDir/run/
$source SetupDAQCommissioning.sh
Then start up the partition.
$setup_daq -p part_jdegenha -d part_jdegenha.data.xml 

Running Athena Stateless Monitoring

Currently the Athena Stateless Monitoring cannot run inside the DAQ partition due to database issues. The next best thing is to run Athena Stateless monitoring separately and have it connect to a running partition. To do this, first get a running partition as stated above. Next, in a new terminal session, cd to
/det/trt/TRTMonitoring/Official_JOs 

Then source the config.sh script, and then run athena:

$source config.sh
$athena AthenaMonitoringStateless_JobOptions.py

Then check in the running partition if there are TRT histograms in the OH.

 

Installing a new version of the TRT monitoring

Changed:
<
<
Running TRT Monitoring online is simply a matter of using the right job options file. The job options file should be found in /sw/atlas/AtlasOffline//.
>
>
Running TRT Monitoring online is simply a matter of using the right job options file. The job options file should be found in /sw/atlas/AtlasOffline//.
  1. create a TESTAREA under /det/trt/TRTMonitoring/. 2. in the TESTAREA create a cmthome directory, and WorkDir:

Revision 112009-01-26 - jdegenha

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 80 to 80
 then
Changed:
<
<
source /sw/CMT/current/mgr/setup.sh
>
>
source /sw/atlas/CMT/v1r20p20080222/mgr/setup.sh
 cmt config
Line: 90 to 90
 cd /det/trt/TRTMonitoring// mkdir source cmthome/setup.sh -tag=,setup,32,runtime,
Added:
>
>
export CORAL_AUTH_PATH=/sw/DbSuppForMx export CORAL_DBLOOKUP_PATH=/sw/DbSuppForMx
 cd WorkDir mkdir run cd run cp /sw/atlas/AtlasTier0//InnerDetector/InDetExample/InDetRecExample/share/jobOptions_cosmic.py .
Added:
>
>
Edit
 jobOptions_cosmic.py 
appropriately to get the monitoring to run.

athena jobOptions_cosmic.py |tee monitoring.log

You should get a root file called

monitoring.root
. Check to make sure that the file has correctly filled histograms.

This is the first step in getting Athena monitoring to run at point1.

 

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)

Revision 102009-01-26 - jdegenha

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 41 to 41
 Part Name: ATLAS Database File: /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml
Added:
>
>

Stand-Alone Partition

Running a stand alone partition is a good way to debug the monitoring framework with the online system. The following describes how to create your own stand along partion and how to run it.
 
Changed:
<
<

Running the Stand-alone partition

>
>

Creating A Stand Alone Partition

Running A Stand Alone Partition

 

Installing a new version of the TRT monitoring

Added:
>
>
Running TRT Monitoring online is simply a matter of using the right job options file. The job options file should be found in /sw/atlas/AtlasOffline//.

1. create a TESTAREA under /det/trt/TRTMonitoring/. 2. in the TESTAREA create a cmthome directory, and WorkDir:

 
mkdir /det/trt/TRTMonitoring/<TESTAREA>/cmthome
mkdir /det/trt/TRTMonitoring/<TESTAREA>/WorkDir
3. in this cmthome area create requirements file:

set CMTSITE STANDALONE
set SITEROOT /sw/atlas
macro ATLAS_DIST_AREA ${SITEROOT}
macro HLT_PROJECT_AREA ${SITEROOT}
macro ATLAS_TEST_AREA /det/trt/TRTMonitoring/<TESTAREA>/WorkDir
macro ATLAS_GROUP_AREA /sw/atlas/<PROJECT>/<RELEASE>
apply_tag groupArea
apply_tag oneGroup
apply_tag oneTest
apply_tag projectArea
macro SITE_PROJECT_AREA ${SITEROOT}
macro EXTERNAL_PROJECT_AREA ${SITEROOT}
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
set CMTCONFIG i686-slc4-gcc34-opt

then

source /sw/CMT/current/mgr/setup.sh 
cmt config

Log out of that session and then log back in.

 
cd /det/trt/TRTMonitoring/<TESTAREA>/
mkdir 
source cmthome/setup.sh -tag=<RELEASE>,setup,32,runtime,<PROJECT>
cd WorkDir
mkdir run
cd run
cp /sw/atlas/AtlasTier0/<RELEASE>/InnerDetector/InDetExample/InDetRecExample/share/jobOptions_cosmic.py .
 

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)

Revision 92008-10-08 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 46 to 46
 

Installing a new version of the TRT monitoring

Added:
>
>

L1-bits and L1-keys

The L1 accepts are given by bits and keys, in addition there are 2 kinds of bits. Decimal numbers used in the trigger details page of the runlist, and the L1 according to emon (eg: [ 0x0 , 0x100000000 , 0x0 , 0x0 ]). To get the mapping from the L1 key to the L1-bit used in emon first, get the mapping from L1-key to L1-decimal bit from the run list / trigger details tab of a recent run. Next (after you set up athena) do ./l1bits4emon #, where # is the L1 decimal bit. l1bits4emon is a script that can be found, (eg: /shared/data/johnda/)
 

Integrating in a new run CASE STUDY (cosmic run 2/10/8)

The monitoring runs ok, out of the box, however some of SFIs are not seeing events from the gather. Problem is that the L1 triggers were configured incorrectly. Need to locate a script (from rhauser) which gives you the L1 bit given the L1 type to configure the monitoring correctly. The script is /atlas-home/1/rhauser/bin/l1bits4emon. The L1 bits are set in the top jobOptions for each monitoring processes, which configure the monitoring process by providing the L1 bit and the

Revision 82008-10-03 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 8 to 8
 Here you are prompted for a hostname, the two TRT Athena monitoring machines are pc-tdq-mon-24 and pc-tdq-mon-25

General structure of the on-line monitoring

Added:
>
>
When the jobOptions files that run the monitoring processes that appear in the DAQ Gui are changed, the oks system does not need to be run agian, the process just needs to be restarted by the run controller. (The OKS only needs to be rerun when there are changes in the xml files)
 

Atlas software at Point 1

The atlas software is located in the directory /sw/atlas/. There you can find the latestest releases that are installed at point 1
Line: 22 to 23
 
  • When running with Atlas partition the log files are stored in (/logs/tdaq-01-09-01/ATLAS)

SFI's and all that jazz

Changed:
<
<
The TRT online monitoring runs after the L2 decision in the SFI
>
>
The L2 decision is made after we get the events from the emon sampler. The online monitoring gets events from the SFI (Sub Farm Input) which basically assembles the full events and then passes them on. the L2 decision is made in SFO's (sub farm output) which are different machines than SFI's.
 

Running the spy GUI

For the Spy GUI, you can do igui_start. Usually you will get it in spy mode, but it can happen that you'll get expert rights. If that happens, go to the "Access Control" menu and choose "Status Display" instead of "Expert Control".
Line: 52 to 53
 Displays the trigger chains for the triggers in the partitions, which L1s feed into which L1 ect. The Trigger Presenter is avialible through the TDAQ panel.

Tips, hints and fun facts

Added:
>
>
  • In order to match a L1 item (eg: L1_EM3) to the bit (eg: decimal number) go to the run list, select a run, and go to the trigger details tag.
 
  • To copy files from lxplus ...
  • To edit this page from outside point 1 you have to go through the proxy setup. Go here and click on proxy setup.

Revision 72008-10-03 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 24 to 24
 

SFI's and all that jazz

The TRT online monitoring runs after the L2 decision in the SFI
Changed:
<
<

Running the spy GUI during a combined run

>
>

Running the spy GUI

For the Spy GUI, you can do igui_start. Usually you will get it in spy mode, but it can happen that you'll get expert rights. If that happens, go to the "Access Control" menu and choose "Status Display" instead of "Expert Control".

Alternatively:

 source the DAQ: source /det/tdaq/scripts/setup_TDAQ.sh

then start the GUI: setup_daq -p ATLAS -d /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml

Added:
>
>

Running the DAQ panel

The DAQ panel lives in /etc/CRD/bin/panel. It's not on every machine, so make sure you're on one of the pc-trt-scr-* or pc-atlas-cr-* machines. (And I think it might even be different between the SCR machines and the atlas-cr machines.) When you open it, you'll need to enter the proper setup script and partition file:

Setup Script: /det/tdaq/scripts/setup_TDAQ_14.2.XY.sh Part Name: ATLAS Database File: /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml

 

Running the Stand-alone partition

Installing a new version of the TRT monitoring

Revision 62008-10-03 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 34 to 34
 

Installing a new version of the TRT monitoring

Integrating in a new run CASE STUDY (cosmic run 2/10/8)

Added:
>
>
The monitoring runs ok, out of the box, however some of SFIs are not seeing events from the gather. Problem is that the L1 triggers were configured incorrectly. Need to locate a script (from rhauser) which gives you the L1 bit given the L1 type to configure the monitoring correctly. The script is /atlas-home/1/rhauser/bin/l1bits4emon. The L1 bits are set in the top jobOptions for each monitoring processes, which configure the monitoring process by providing the L1 bit and the

Trigger Presenter

Displays the trigger chains for the triggers in the partitions, which L1s feed into which L1 ect. The Trigger Presenter is avialible through the TDAQ panel.
 

Tips, hints and fun facts

  • To copy files from lxplus ...

Revision 52008-10-02 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 24 to 24
 

SFI's and all that jazz

The TRT online monitoring runs after the L2 decision in the SFI
Added:
>
>

Running the spy GUI during a combined run

source the DAQ: source /det/tdaq/scripts/setup_TDAQ.sh

then start the GUI: setup_daq -p ATLAS -d /db/tdaq-01-09-01/combined/partitions/ATLAS.data.xml

 

Running the Stand-alone partition

Installing a new version of the TRT monitoring

Revision 42008-10-02 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 19 to 19
 

Log files

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) on the computer where the process is running. (Where is this configured???) To determine the computer on which the process is running
Changed:
<
<
  • When running with Atlas partition the log files are stored in (???) (Where is this configured???)
>
>
  • When running with Atlas partition the log files are stored in (/logs/tdaq-01-09-01/ATLAS)
 

SFI's and all that jazz

The TRT online monitoring runs after the L2 decision in the SFI

Revision 32008-10-02 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
-- JohnAlison - 02 Oct 2008

TRT Expert Monitoring

Line: 9 to 9
 

General structure of the on-line monitoring

Deleted:
<
<

log file

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) (Where is this configured???)
  • When running with Atlas partition the log files are stored in (???) (Where is this configured???)

SFI's and all that jazz

 

Atlas software at Point 1

The atlas software is located in the directory /sw/atlas/. There you can find the latestest releases that are installed at point 1
Line: 23 to 17
 

The TRT Monitoring segment for the combined run

The segment for the combined run is located in /db/tdaq-01-09-01/trt/segments/Monitoring/ (where 01-09-01 is the latest version of the tdaq)
Added:
>
>

Log files

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) on the computer where the process is running. (Where is this configured???) To determine the computer on which the process is running
  • When running with Atlas partition the log files are stored in (???) (Where is this configured???)

SFI's and all that jazz

The TRT online monitoring runs after the L2 decision in the SFI
 

Running the Stand-alone partition

Revision 22008-10-02 - johnda

Line: 1 to 1
 
META TOPICPARENT name="AtlasOperation"
Changed:
<
<
-- Main.johnda - 02 Oct 2008
>
>
-- JohnAlison - 02 Oct 2008
 

TRT Expert Monitoring

Added:
>
>

Connecting to the Point 1 network

 To connect to point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh atlasgw.cern.ch.
Added:
>
>
Here you are prompted for a hostname, the two TRT Athena monitoring machines are pc-tdq-mon-24 and pc-tdq-mon-25

General structure of the on-line monitoring

log file

  • When running the standalone partition the log files are stored in (/tmp/[PartitionName]) (Where is this configured???)
  • When running with Atlas partition the log files are stored in (???) (Where is this configured???)

SFI's and all that jazz

Atlas software at Point 1

The atlas software is located in the directory /sw/atlas/. There you can find the latestest releases that are installed at point 1

Database access at Point 1

The TRT Monitoring segment for the combined run

The segment for the combined run is located in /db/tdaq-01-09-01/trt/segments/Monitoring/ (where 01-09-01 is the latest version of the tdaq)

Running the Stand-alone partition

Installing a new version of the TRT monitoring

Integrating in a new run CASE STUDY (cosmic run 2/10/8)

 
Changed:
<
<
Here you are prompted for a hostname viable options include: pc-tdq-mon-24 and pc-tdq-mon-25
>
>

Tips, hints and fun facts

  • To copy files from lxplus ...
  • To edit this page from outside point 1 you have to go through the proxy setup. Go here and click on proxy setup.

Revision 12008-10-02 - johnda

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="AtlasOperation"

-- Main.johnda - 02 Oct 2008

TRT Expert Monitoring

To connect to point 1 from outside the network. Login to lxplus and then ssh through the gateway: ssh atlasgw.cern.ch.

Here you are prompted for a hostname viable options include: pc-tdq-mon-24 and pc-tdq-mon-25

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ATLAS?Please contact the page author (see Topic revision above) or the Run Coordinator of the specific system.
Contact SysAdmins support only for technical issues