Page Comparison

In some occasions, the analytics team may need to take specific actions to check or update changes in the datamarts. Examples in which this may happen may include the following:

...

1. If refreshes aren't showing updated data, it may be because the sample ETL is down. If LIMS team has done their part, but the bsp.analytics_sample and analytics.bsp_sample DMs are not updated, check the ETL agent here: http://analytics:8090/etl_runs?status=Succeeded&status=Failed&status=Running&status=Abandoned&agent=analytics.tiger.agents.BspSample&action=Refresh

2. PDO STAR, RGHQS, and Read Group Metadata Datamart Refresh Push

...

2.1. Option 1

...

- Refresh PDOSTAR & RGHQS using RunEtl tool

1. Instructions here

2.2. Option 2 - Manual Refresh in PDO STAR using a query (

...

alternate method using "Old" Collaborator Sample IDs)

1. Log in to your favorite SSH tool (e.g. SecureCRT)
   
   1. connect to "analytics" server (command: ssh analytics)
   2. enter the following: /home/unix/analytics/TigerETL3/runEtlShell.sh

   3. paste & run the following etl-command separately for each PDO, changing PDO-12480 to the relevant PDO returned above.once, pasting the "Old" Collaborator Sample IDs in the query below

      Code Block
      language scala
      import analytics.tiger.agents.PdoStar._ AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", deltaFromSqlcollSamplesToPdoSamples(List("""SELECT DISTINCT a.pdo_name||','||a.pdo_sample_id FROM cognos.pdo_star5 a WHERE a.pdo_name = 'PDO-12480'"""oldcollabSMID1","oldcollabSMID2")) flatMap DataSetEtl() flatMap analytics.tiger.agents.PdoStar.pipeline)

      The NOTE: These are supposed to be two lines (the first ends with an underscore). Once you hit enter, the command takes a little while. You'll know it's complete when you get the scala prompt again.

2.

...

3. Option

...

3 - Manual Refresh in PDO STAR using a query (

...

using PDO_NAMEs)

1. Log in to your favorite SSH tool (e.g. SecureCRT)
   
   1. connect to "analytics" server (command: ssh analytics)
   2. enter the following: /home/unix/analytics/TigerETL3/runEtlShell.sh

   3. paste & run the following etl-command once, pasting the "Old" Collaborator Sample IDs in the query belowseparately for each PDO, changing PDO-12480 to the relevant PDO returned above.

      Code Block
      language scala
      import analytics.tiger.agents.PdoStar._ AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", collSamplesToPdoSamplesdeltaFromSql(List(""oldcollabSMID1","oldcollabSMID2")"SELECT DISTINCT a.pdo_name||','||a.pdo_sample_id FROM cognos.pdo_star5 a WHERE a.pdo_name = 'PDO-12480'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.PdoStar.pipeline)

      
      

   4. NOTE: These are supposed to be two lines (the first ends with an underscore). Once you hit enter, the The command takes a little while. You'll know it's complete when you get the scala prompt again.

2.

...

4. Reviewing that PDO STAR did refresh

1. Rerun the PDO_STAR query above. This refresh is immediate, so you can run the query right away.
2. If it returns nothing, the DM is all set.
3. If it still returns records, ensure that that is unexpected behavior and try the refresh again. (occasionally, there are other samples that need to remain with the "old" IDs, as is usually the case in sample swaps)
   1. If it still returns unexpected records, talk with Nasko.

2.

...

5. Refreshing the ReadgroupMetadata DM in SEQPROD Database

...