| Gliffy | ||
|---|---|---|
|
...
INSERT INTO bsp_plating_special_refreshes VALUES('6E3K2', SYSDATE);
COMMIT;
Note: There is no "SM-" prefix,
...
Issue/Error: Once a week (1am on Fri) PdoStar5 goes through 1y-refresh. Sometime you might want to trigger it earlier.
Diagnosis:
Remedy: Connect to "analytics" server using your favorite SSH tool and run the following line:
...
- Seq Only PDOs do not get BAM files in PDO_STAR5_Aux DM
- Plated sample in BSP_Plated_Samples is different than slxre_readgroup_metadata.sample_id
- Read group of the PDO Sample is missing in SLXRE2_RGHQS, most likely because there isn't a record in metrics.basecall_analysis for metrics_type Indexed or Unindexed.
- PM billed sample before we refreshed PDOSTAR5 picked the most recent aggregation or before sample met full coverage
...
- None
- Check if Mercury DWH and Mercury Run API still do not agree about the plated sample. If they don't - seek help from Mercury team. If they match, a refresh of the flowcell should correct it. BSP exported samples might be associated with the wrong PDO in BSP_Plated_samples because the mapping in BSP is not accurate. If it is important to fix the data , BSP team could help and then request a refresh for the affected plated sample as instructed in the recipe on this page to update BSP Plating DM. Finally, make sure to refresh in PDO Star5 if it won't happen automatically.
- Make ticket for the PIpeline and refer to PO-13231 as being the same issue.
- PDOs created within past year will get refreshed on Friday 2:55 am or Analytics manually refresh for faster data delivery. PDOs older than 1 year MUST be manually refreshed.
Issue: When upstream data change, some samples in ReadgroupMetadata might need to be refreshed.
...
Note: No COMMIT needed in this particular call - it's implicitly done for you. Just replace the XYZ with a flowcell barcode. Leave all other parameters there and unchanged.
Issue: How to prevent an entire FC from getting into ReadgroupMetadata (Analytics Blacklisting).
Remedy: Follow the instructions here for blacklist runfolder from RunMercury/RghqsLoader agents
Note: Blacklist_timestamp is not null for blacklisted flowcells.
Issue: How to whitelist a FC for processing by Run Mercury and RGHQS agents.
Remedy: Follow the instructions here for whitelist runfolder from RunMercury/RghqsLoader agents
Issue: How to refresh Picard Aggregation DMs for a specific set of samples. Note, this will not refresh any metrics unless they are part of a new aggregation (new version). Only basecall metrics rolled up at library and sample level get updated
Remedy: Follow the instructions here for Manual Refresh of Picard Aggregations for a list of samples.
Issue: PicardAggregator ETL fails with (COGNOS.SLXRE2_PAGG_SAMPLE_PK) violated error.
Cause: Most likely Is_latest flag is set for more than 1 aggregations.
Remedy: Find the sample that cause it (run the query below) and make a blocker PO ticket + notify somebody from Pipeline team to address it ASAP because the ETL won't recover until this is fixed.
| Code Block |
|---|
SELECT DISTINCT SAMPLE FROM (
SELECT a.project, a.SAMPLE, a.LIBRARY, a.data_type
FROM metrics.aggregation a
WHERE nvl(a.modified_at, a.workflow_end_date) >= to_date(<START Date from ETL dashboard>, 'yyyy-Mon-dd hh24:mi:ss')
AND nvl(a.modified_at, a.workflow_end_date) < to_date(<END Date from ETL dashboard>, 'yyyy-Mon-dd hh24:mi:ss')
AND a.is_latest <>0
GROUP BY a.project, a.SAMPLE, a.LIBRARY, a.data_type
HAVING count(*)>1
) |
...
Remedy: Contact Tammy, Andrew B. , Mike DaSilva or John W. to clear the bad run from their dashboard. Designation ETL will recover as soon as the run is cleared.
Issue: Scala ETL ERROR: analytics.etl.DeckEventAgent$ (Most likely 'lcSet' tag is missing)
Diagnosis: Sometimes this Squid web-service returns no LcSet. Comment about this incident is automatically posted in IPI-61224 and ticket reopened.
Remedy: Just follow up with LIMS team. Our DeckEventAgent will catch up as soon as LIMS service is fixed. No other intervention is needed.
Issue: analytics.etl.PkViolationWarning. unique constraint (COGNOS.RGHQS_METADATA_PK) violated
Diagnosis: This happens when 2+ readgroups having same (fc, lane, mol_index) try to get into datamart. Picard calls it "CLASH". RunMercuryAgent would ignore abusing readgroups and move on with the rest of 'good' ones. Email would give you detailed troubleshooting info.
Remedy: There isn't much for us to do. If upstream issues are resolved you can request a ReadgroupMetadata refresh for the whole run/fc (see above). You can search for the RG in GPINFOJIRA by searching the FC ID to see if it has been blacklisted and already handled upstream.
Issue/Error: ETL: DB error, server: (thorium or vorarestore), Error: ORA-12008: error in materialized view refresh path, ORA-25153: Temporary Tablespace is Empty
Diagnosis: This happens when the DB team is doing some kind of backup or transfer.
Remedy: These errors can be ignored. Everything is actually fine.
...