Future topics

...

• MM - Tentative dates Nov 22,30  Dec 7,14,21BCN - Feb 19-22, Mar 21-27
• CG - 2024 Dec 16, 5, 8, 11, 15, 20, 21, Jan 513, 19-20
  

Unassigned RPT tickets: Link to filter in Jira

2023-07-13

• Monitor error and warning emails - how to capture errors that need immediate attention.
• Jira admin flash training - issues like

  Jira Legacy
  server GP-broad JIRA serverId de1bf0f9-3cc7-3c47-be9e-dc2fbe05975b key GPSM-47

2023-07-13

• Tableau server - negotiated new license model. We keep current configuration - 8 core server with unlimited number of Explorers. Still would like to clean up the server from inactive users and generally, host only GP stuff and users.
• Covid ETLs and extracts were stopped. We would need to retire some reports and/or replace them with new static boards. Waiting for heads up from Heather.
• Quality System revamp - on hold new requests for reporting on new operational metric.
• Quarterly goals were just posted. Still to figure out which ones will need our attention. Watch out for Clinical BGE, DRAGEN for NVX, Clinical WGS, Walkup data to buckets (replace Walkup portal with SIDR)

...

Zach told Jim we were worried about ExomesOnTheCloud. We should reach out to GreenTeam to ensure we know what is happening when.

Tina's email: Associating Affected Sample (downstream data) to related BQMS events upstream – there was a meeting about this. It's too unregulated right now. Simplest way may be to go up to the root and interrogate its whole tree for BQMS-related tickets, and let the user figure it out. Jim asked the lab to come up with a broader vision & scope before we build anything.

Sam: 

2018-02-22

News about metrics on the cloud - Rachel heard rumors that Exomes on the Cloud might be on hold, but we have no confirmation. There is a ticket to document how aggregations work to help Mercury, which might be helpful for us as well (

...

Tina's  

Tina's email: Associating Affected Samples to related BQMS Tickets - Amy started an early assessment

• AB: ~30% of tickets don't actually have samples listed, so we'll have to consider how to treat a blank Sample ID field plus (say) "Samples affected = 2" but is linked to a 96 sample LCSET.

Zach presented Grant View report to Stanley Center and other groups. They loved it. Trying to push back on implementing different versions of it for every group regardless of its size.

Rachel reported that miss-communication between requestors , Analytics and OPs sometimes get in a way of quicker resolution of re-patienting requests.

2018-02-15

Don't simply say we can do something in JIRA/Tableau if Mercury might be the better place!

...

Fargo outage Sept 1-5, no access to /seq/fargo_picard_aggregation (doesn't affect us)

Ruthless culling of Tableau Server has begun. EVERYONE: Go through Archive and delete your reports, back up offline what you want to keep.

...

• Request, API, adaptor
  • 1) Request can be articulated in different ways (e.g. URL, ETL etc)
  • 2) new tool produces a scala stream output. It's client-agnostic. 
  • 3) Scala output is adapted to desired format by one of a set of adaptors, determined by how the request was sent.
• Fields can be manipulated as in v1, but w/ different syntax: Existing API queries will need to be ported and their syntax adapted.
  
  • Aqua: starting field. Dotted line: assumed type. Straight line: specified transformation. Grey field: final options
  • All fields are exported in string format, so non-strings require syntax specs. 
  • New option:TO:BITS 
  • New option: strip HTML (though html for links can be useful)
  • APIv3 idea : auto-transform to the original JIRA field formats
• Still have the thousand-record input limit from JIRA
• Zach will test it on Skylab , then we can plan a total port-over

2016-12-15

• code review for arrays: Nasko will help look at the query. Chris will help look at whether a published datasource might help.
• consent withdrawal: New slack room pt-consent-withdrawal. Betty likes the idea of this conversation.
• TIGER ETL can be used to turn arrays code views into tables once they're solidified
• SAP changes (out of quoteserver in a few months). Pricing easier will help make it easier to work with companies for projects more insulated from NIH changes.
  • 
    

2016-12-08

• Offices
• Sample removal of consent: owning it, doing it, whole tree, etc. (SUPPORT-2345) 
  • Maybe a new ticket type with all the groups in the workflow? Let's get comfortable with the process. What tickets cover it? (RPT-3551)
  • How do other seq centers deal with this? Let's ask Betty.
  • Slack group for figuring it out?
  • KC will pull together preliminary info
• Tableau Server VM migration from Somerville cluster to Boston cluster: really complex and ugly, but Tina has been helping a lot. 
  • Friday 4pm. Turn it off, unlicense it, move it, license it on the new hardware.
  • TS-beta seems better now – maybe we'll see an overall performance improvement?
  • lab/tableau is going away. New user charles/tableau_auth
    • Need to make this change everywhere.
    • Tableau SAP situation is complex too, so we had to make changes there.
  • Next we'll push for more RAM so we don't run out on Wednesdays
• Job families: Analytics Engineering & Software Engineering job families
• Field trip Amy & Zach went to FC land. Tammy uses many Gdocs for designations. Still major room for improvement here.
• Nasko building yet another JIRA interface (more powerful, easier to use – explode etc)

Zach: Dave Zdeb: preliminary results on pooling are good - we might reduce lanes and therefore reagent costs (which are higher than salaries).

2016-12-01

Amy helping seq lab with designations (to get their work started quicker). For ExEx FCs Tammy prints the JIRA ticket and lab manually types the ticket key to get Hamilton messaged correctly. Amy added a barcode for scanning that onto the Hamilton. 

• Next step might be identifying the new most problematic step in the workflow. Zach & Amy can go watch (this happens every day). They may need more scanners.
• The recent FC swap problem was complex and theoretically could happen again.

JIRA ran out of DB connections with low memory this morning: lots of groups were doing bulk ticket transitions (SRS many consolidation tix, etc). Erik restarted (no DaveG DB intervention required). 

...

Yesterday's career brainstorming session was just awful

2016-11-23

TC16 interesting talks

• Zach: Florida Hospital (for RFID tracking, rigorous process implementation, process improvement and result - more than Tableau details)
• Zach: two developer portal programs by/for Tableau 10: Scout (assessing workbook complexity, usability, performance) & Auditor (performance)

...

• unused/outdated datasources
• Steve has too many unused worksheets & datasources hidden in his workbooks

2016-11-17

No meeting: too many people out.

2016-11-10

No meeting: Tableau conference

2016-11-03

No meeting: "Quiet week."

2016-10-27

Arrays: we're on track. Still need a bit from Mercury team. Still planning a few things. We can expect real data & final validation in about a month.

FP: Jim says we'll have a decision of sorts by the beginning of next quarter. JonT will build either a Fat Panda workflow or a rebuild of GAP for Fluidigm.

Mariela's busy with user requests and such.

Amy's an admin on cloud JIRA without permission to act as an admin at this point.

2016-10-20

FP maybe working again. Same story: Not under control.

Merc user meeting yesterday: Dennis, Eliot, Dave, John all presented new features for the next 2 releases (Marcia testing all of it)

• SAP stuff maybe in the next release
• The big push is Arrays, ten SAP (two major things at once again here)

Mariela demo of Pass through Tableau functions preference over LOD. NFC LOD Custom Query in Daily FC Tracker Performance Demo

...

Nasko cloned BSP.ANALYTICS_SAMPLE to ANALYTICS.BSP_SAMPLE (no latency)

2016-10-13

Sarah Calvo course looks interesting but is WAY oversubscribed

• Update: Videos available: 
  • Class 1: Central dogma https://vimeo.com/188213623/30db00934c
  • Class 2: Sequence variation & inheritance https://vimeo.com/190197861/a34fe76866
  • Class 3: Origin of life and basics of cell biology https://vimeo.com/190860151/be1c97762b
  • Class 4: Gene expression and basic metabolism https://vimeo.com/192126809/7db3ade842

• FP on Fire again!!! 
  • Fluidigm is coming in to help.
  • Fat Panda optimism is unhindered
• MEGA Chip maybe fixed but there's a problem with the data not showing updated (new autocall not showing in reports/LIMS reports)
• ARRAYS
  • Green Team "done" end October
  • We'll need a plan for data crosscheck etc.
• DATABASE PROBLEMS
• LabOpsJiraTest schema didn't get copied to dev correctly, there's a bunch of stuff we have to deal with

Ancestry.com (we do limited genome stuff for them) is so happy with our work, they'll keep sending to us

"Secretly rolling in a bunch of new loaner sequencers" (TimD) (secret capacity)

• 150/day more?
  
• Do we have the analysis capacity for this? We've had some days of near-zero in cloud analysis?

Thurs Nov 3 BTUG here

Mariela will be on health leave Nov 14 - newyear

Chris MKD updates with Hayley

Chris optimizing Stanley Center sample mgmt code using Mariela suggestions. Lots of eyes. Case/Control parsing is tricky

• Discuss with Mercury team 

Whenever possible, we want to spend our energy fixing fundamental workflow problems rather than creating error reports (e.g. designations, "enabling" problems)

• Mexico idea: PDOs that haven't quite been closed out and what samples need to get hurried
• Reinventing the wheel

...

KC all arrays all the time

2016-10-06

Fingerprinting situation:

• Fire #2 fixed again, but wasn’t clear why (just like Fire#1) (so probably not for long!)
• Fat Panda isn't working at all right now, so they aren't yet setting up the automation

WGS team waiting for a Mercury fix and a RapidQC fix before submitting another LCSET

Big revenue month (Sheila: “we’re doing really well financially”) (who is "we"???)

Arrays: moving forward nicely. We'll add Analytics schema to SEQDEV3, and we should ask Dave to include that in the Prod/Dev backups.

• last night the spark agent in the cloud DB got stalled, so Nasko will fix it.

Amy demoed "Create and Link multiple tickets" feature she used in the CCLF JIRA using a new feature in the C&L plug-in

...

...