In some occasions, the analytics team may need to take specific actions to check or update changes in the datamarts. Examples in which this may happen may include the following:
...
| Info |
|---|
PDO STAR automatically refreshes samples that are not billed every two hours, and does a full refresh of 1-year data on Friday mornings at 2:55 am. If metadata change requests are for orders that are not billed or are less than a year old, a manual push may not be necessary. However, step 2.4 below should be followed regardless to check whether the metadata changes were made in PDO STAR. |
2.1. PDOSTAR refresh Option 1 -
...
Using RunEtl tool (Also RGHQS refresh)
- Instructions here
2.2. PDOSTAR refresh Option 2 - UNIX
...
, using a query (
...
using "Old" Collaborator Sample IDs)
| Note | ||
|---|---|---|
| ||
For this process, you can bypass section 2.1 and use the "Old" Collaborator Sample IDs obtained in section 1. |
- Log in to your favorite SSH tool (e.g. SecureCRT)
- connect to "analytics" server
- enter the following: /home/unix/analytics/TigerETL3/runEtlShell.sh
paste & run the following etl-command once, pasting the "Old" Collaborator Sample IDs in the query below
Code Block language scala import analytics.tiger.agents.PdoStar._ AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", collSamplesToPdoSamples(List("oldcollabSMID1","oldcollabSMID2")) flatMap DataSetEtl() flatMap pipeline)NOTE: These are supposed to be two lines (the first ends with an underscore). Once you hit enter, the command takes a little while. You'll know it's complete when you get the scala prompt again.
2.3. PDOSTAR refresh Option 3 - UNIX, refresh
...
using a query (using PDO_NAMEs)
| Note | ||
|---|---|---|
| ||
In order to follow this process, you need to obtain the PDO_NAME of any records returned in section 1.1. |
- Log in to your favorite SSH tool (e.g. SecureCRT)
- connect to "analytics" server (command: ssh analytics)
- enter the following: /home/unix/analytics/TigerETL3/runEtlShell.sh
paste & run the etl-command separately for each PDO, changing PDO-12480 to the relevant PDO returned above.
Code Block language scala AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", deltaFromSql("""SELECT DISTINCT a.pdo_name||','||a.pdo_sample_id FROM cognos.pdo_star5 a WHERE a.pdo_name = 'PDO-12480'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.PdoStar.pipeline)The command takes a little while. You'll know it's complete when you get the scala prompt again
2.4.
...
PDO STAR review to ensure it did refresh
- Rerun the PDO_STAR query above. This refresh is immediate, so you can run the query right away.
- If it returns nothing, the DM is all set.
- If it still returns records, ensure that that is unexpected behavior and try the refresh again. (occasionally, there are other samples that need to remain with the "old" IDs, as is usually the case in sample swaps)
- If it still returns unexpected records, talk with Nasko.
2.5.
...
Refresh the ReadgroupMetadata DM in SEQPROD Database
| Tip | ||||
|---|---|---|---|---|
| ||||
Review Analytics ETL Framework for more details on this section |
...