/
Tiger3 ETL Shell Commands (refreshes, etc.)
Tiger3 ETL Shell Commands (refreshes, etc.)
- Atanas Mihalev
- Mariela Mihaleva
- Rafael Sanchez (Deactivated)
- Kristen Connolly (Unlicensed)
Owned by Atanas Mihalev
- ssh-connect to "analytics" server using your favorite SSH tool. For Windows SecureCRT is offered as standard ssh-software to Broadies. For Mac you can connect through the command line in Terminal.
- Run etlShell
/home/unix/analytics/TigerETL3/runEtlShell.sh
- run etl-commands as you see fit
ETL Tasks
| TASK description | COMMAND |
|---|---|
Manually push into GitHub by providing CSV-list of dbObjects and optional commit message. Note that dbObjects are specified by (dbname.schema.objectname) triplets where dbname is: seqprod, labopsjira or bsp If no explicit commit-message is provided then built-into-your-code comments would be picked /* GITHUBCOMMENT: RPT-1234 */ | analytics.tiger.agents.OracleVCS.main(Array("labopsjira.REPORTING.ZZZAB_METADATA_TRANSITIONS,seqprod.COGNOS.ACTIVE_LAB", "Monads are great"))
|
| blacklist runfolder from RunMercury/RghqsLoader agents | CognosDB("RunfolderAPI.blacklist", RunfolderAPI.blacklistRun("runfolder1,runfolder2"))
|
| whitelist runfolder from RunMercury/RghqsLoader agents | CognosDB("RunfolderAPI.whitelist", RunfolderAPI.whitelistRun("runfolder1,runfolder2"))
|
| Manual Refresh in PDO Star5 using a query (using PDO_NAMEs) | AnalyticsEtlDB("analytics.tiger.agents.PdoStar.fromSQL", deltaFromSql("""SELECT DISTINCT a.pdo_name||','||a.pdo_sample_id FROM cognos.pdo_star5 a WHERE a.pdo_name = 'PDO-14680'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.PdoStar.pipeline)
|
| Manual Refresh in PDO Star5 using a query (alternate method using "Old" Collaborator Sample IDs) | DEPRECATED (use RunETL tool instead) import analytics.tiger.agents.PdoStar._
AnalyticsEtlDB("analytics.tiger.agents.PdoStar.CollaboratorSamples", collSamplesToPdoSamples(List("oldcollabSMID1","oldcollabSMID2")) flatMap DataSetEtl() flatMap pipeline)
|
| Manual Refresh of Picard Aggregations for a list of samples (Research) | DEPRECATED (use RunETL tool instead) import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.CognosDB("analytics.tiger.agents.PicardAggregator.Samples", regSamples("JB-0236-T-02,TCGA-HU-A4H5-11A-11D-A25E-08"))
|
| Manual Refresh of Picard Aggregations using a query (Research) | import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.CognosDB("analytics.tiger.agents.PicardAggregator.FromSql", regFromSql("SELECT a.SAMPLE FROM cognos.slxre2_pagg_sample a WHERE a.SAMPLE = 'NWD725917'"))
|
| Manual Refresh of Picard Aggregations for a list of samples (CRSP) | DEPRECATED (use RunETL tool instead) import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.AnalyticsEtlDB("analytics.tiger.agents.PicardAggregator.CRSP.Samples", crspSamples("SM-INAAG,SM-INAAL"))
|
| Manual Refresh of Picard Aggregations using a query (CRSP) | import analytics.tiger.agents.PicardAggregator._
analytics.tiger.utils.AnalyticsEtlDB("analytics.tiger.agents.PicardAggregator.CRSP.FromSql", crspFromSql("SELECT a.SAMPLE FROM cognos.slxre2_pagg_sample a WHERE a.SAMPLE = 'SM-INAAL'"))
|
| Manual Refresh of RapidQC Aggregations for a list of samples | CognosDB("analytics.tiger.agents.RapidQcAggregator.Manual", deltaETL(Set("NWD581810")) flatMap DataSetEtl() flatMap analytics.tiger.agents.RapidQcAggregator.mainPipeline)
|
| Manual Refresh of RapidQC Aggregations using a query | CognosDB("analytics.tiger.agents.RapidQcAggregator.fromSQL", deltaFromSql("""SELECT a.bsp_collaborator_sample_id FROM pdo_star5_aux a WHERE a.pdo_sample = 'SM-HQV67'""") flatMap DataSetEtl() flatMap analytics.tiger.agents.RapidQcAggregator.mainPipeline)
|