Versions Compared

Version Old Version 35 New Version Current
Changes made by

Kristen Connolly (Unlicensed)

Atanas Mihalev

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
  1. ssh-connect to "analytics" server using your favorite SSH tool. For Windows SecureCRT is offered as standard ssh-software to Broadies. For Mac you can connect through the command line in Terminal.


  2. Code Block
    languagebash
    themeEmacs
    titleRun etlShell
    /home/unix/analytics/TigerETL3/runEtlShell.sh


  3. run etl-commands as you see fit

...

TASK descriptionCOMMAND

Manually push into GitHub by providing CSV-list of dbObjects and optional commit message. Note that dbObjects are specified by (dbname.schema.objectname) triplets where dbname is: seqprod, labopsjira or bsp

If no explicit commit-message is provided then built-into-your-code comments would be picked

/* GITHUBCOMMENT: RPT-1234 */



Code Block
languagescala
analytics.tiger.agents.OracleVCS.main(Array("labopsjira.REPORTING.ZZZAB_METADATA_TRANSITIONS,seqprod.COGNOS.ACTIVE_LAB", "Monads are great"))


blacklist runfolder from RunMercury/RghqsLoader agents


Code Block
languagescala
CognosDB("RunfolderAPI.blacklist", RunfolderAPI.blacklistRun("runfolder1,runfolder2"))


whitelist runfolder from RunMercury/RghqsLoader agents


Code Block
languagescala
CognosDB("RunfolderAPI.whitelist", RunfolderAPI.whitelistRun("runfolder1,runfolder2"))


Manual Refresh in PDO Star5 using a query (using PDO_NAMEs)


Code Block
languagescala
AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", deltaFromSql("""SELECT DISTINCT a.pdo_name||','||a.pdo_sample_id FROM cognos.pdo_star5 a WHERE a.pdo_name = 'PDO-14680'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.PdoStar.pipeline)


Manual Refresh in PDO Star5 using a query (alternate method using "Old" Collaborator Sample IDs)

DEPRECATED (use RunETL tool instead)

Code Block
languagescala
import analytics.tiger.agents.PdoStar._
AnalyticsEtlDB("analytics.tiger.agents.PdoStar.
fromSQL
CollaboratorSamples", collSamplesToPdoSamples(List("oldcollabSMID1","oldcollabSMID2")) flatMap DataSetEtl() flatMap pipeline)


Manual Refresh of Picard Aggregations for a list of samples (Research)

DEPRECATED (use RunETL tool instead)

Code Block
languagescala
import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.CognosDB("analytics.tiger.agents.PicardAggregator.
Manual
Samples",
deltaETL
regSamples(
Set(
"
ALCH
JB-
ABSQ
0236-
TTP1-A-12-0-D-A489-36","ALCH-ABSQ-NB1-A-1-0-D-A489-36
T-02,TCGA-HU-A4H5-11A-11D-A25E-08"))
flatMap
DataSetEtl() flatMap analytics.tiger.agents.PicardAggregator.Regular.pipeline)

 Manual Refresh of Picard Aggregations using a query (Research)

 Code Block
languagescala
import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.CognosDB("analytics.tiger.agents.PicardAggregator.
fromSQL
FromSql",
 deltaFromSql
 regFromSql("
""
SELECT a.SAMPLE 
FROM cognos
FROM cognos.slxre2_pagg_sample a WHERE a.SAMPLE = '
CAD-JDV-098
NWD725917'"
""
)
 flatMap DataSetEtl() flatMap analytics.tiger.agents.PicardAggregator.Regular.pipeline
)

Manual Refresh of Picard Aggregations for a list of samples (CRSP)
DEPRECATED (use RunETL tool instead)
 Code Block
languagescala
import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.AnalyticsEtlDB("analytics.tiger.agents.PicardAggregator.CRSP.
Manual
Samples", 
deltaETL
crspSamples(
Set(
"SM-
HXT86
INAAG,SM-INAAL"
)
)
 flatMap DataSetEtl(
)
 flatMap analytics.tiger.agents.PicardAggregator.CRSP.pipeline)

Manual Refresh of Picard Aggregations using a query (CRSP)

 Code Block
languagescala
import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.AnalyticsEtlDB("analytics.tiger.agents.PicardAggregator.CRSP.fromSQLFromSql", deltaFromSql crspFromSql("""SELECT a.SAMPLE FROM cognos.slxre2_pagg_sample a WHERE a.SAMPLE = 'SM-HUTACINAAL'""")) flatMap
DataSetEtl()
flatMap analytics.tiger.agents.PicardAggregator.CRSP.pipeline)

Manual Refresh of RapidQC Aggregations for a list of samples

 Code Block
CognosDB("analytics.tiger.agents.RapidQcAggregator.Manual", deltaETL(Set("NWD581810")) flatMap DataSetEtl() flatMap analytics.tiger.agents.RapidQcAggregator.mainPipeline)

Manual Refresh of RapidQC Aggregations using a query

 Code Block
CognosDB("analytics.tiger.agents.RapidQcAggregator.fromSQL", deltaFromSql("""SELECT a.bsp_collaborator_sample_id FROM pdo_star5_aux a WHERE a.pdo_sample = 'SM-HQV67'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.RapidQcAggregator.mainPipeline)

...