Page Comparison

Summary of sample changes (see the comprehensive samples report for more details):

1. Per TCGA this new format is not backwards compatible with the previous XML, so if you use a parser customized to the previous XSD, it may not work correctly.  This change is confined to the XML, no other TCGA data is affected by this change.
2. These changes are detailed at https://wiki.nci.nih.gov/display/TCGA/Release+1.39.1#Release1.39.1-XSD2.7Implementation
3. In the new format, 'CQCF' parameters are removed. Instead, they are provided in new XML format, SSF.
4. According to this change, 'cqcf' parameters are removed and "patient.tumor_samples" and "patient.normal_controls" parameters are added in the clinical merged data. 
5. Added parameters of two new XML formats, OMF and SSF in the clinical merged data 

1. Loaded this data run
2. Numerous internal updates to further simplify deployment, which e.g. will help provision AWG-specific databases
3. The HeartBeat API function now includes: the directory from and time at which the API server was launched
4. Dropped beta clause from version

Summary of sample changes (see the comprehensive samples report for more details):

Extensive improvements to clinical data, in that Clinical_Pick_Tier1 archives now bundle 2 forms of values:

1. Entire set of TCGA CDEs, verbatim (in new All_CDES.txt file): adding over 700 additional clinical parameters

2. In addition to the CDE subset normalized by Firehose for downstream analyses (in <cohort>.clin.merged.picked.txt file)

3. For example, to date the ACC picked file has contained less than 20 CDEs while All_CDEs.txt now contains more than 100.
4. Followup values are merged, when available, to yield the most up-to-date values per CDE
5. Corrected problem wherein some True/False values for regimen_indication CDE were erroneously swapped
6. Created an interactive table CDEs, which on a single page shows exactly what CDEs are selected for analyses in Firehose for all disease cohorts. Updated the FireBrowse clinical samples API to reference this new CDE table
7. Enhanced Merge_Clinical pipeline to leverage auxiliary CDEs when available (COAD, READ, ESCA): for all primary CDEs that also have a value in the aux CDE file (e.g. MSI), we now replace the primary value if it is NA and the aux value is not NA

1. New MAF for Diffuse Large B-cell Lymphoma (DLBC, 48 mutation samples)
2. Oncotator now included in Firehose mutation pipelines, to standardize TCGA MAFs to a common format:
   1. This substantially improves the consistency and utility of TCGA MAFs
   2. hg18 MAFs lifted over to hg19
   3. All MAFs re-annotated against Gencode v19
   4. Oft-requested custom columns, such as amino acid change, now present in all MAFs
      
   5. Oncotated MAFs are available in 2 pipeline output archives
      1. Mutation_Packager_Oncotated_Calls
      2. Mutation_Packager_Oncotated_Raw_Calls
      reflecting the separation of MAFs into two sets (raw/automated and curated, per the Spring 2015 analysis run)

1. Merge_protein_exp__mda_rppa_core__mdanderson_org__Level_3__protein_normalization__data
   1. Validation and animal source suffixes now stripped off of antibody name to account for several new batches that no longer include them
   2. Now returns a union of antibody names from all samples, rather than failing when all samples don't have the exact same antibodies
      1. This does not fix the below RPPA issues when different names for the same antibodies are used (e.g. Acetyl-a-Tubulin-Lys40/Acetyl-a-Tubulin(Lys40), ARHI/DIRAS3) - in these cases each antibody name will appear on separate lines of the merged files until a fix is made at MDACC.
      2. This enables RPPA analysis for aggregate cohorts such as STES, KIPAN, and GBMLGG
2. RPPA_AnnotateWithGene: normalizes the antibody reference files provided by MDACC into a two-column tsv with standardized header, gene name, and antibody name (stripped of suffixes). This file is now provided in the archive.

1. Ingest this August 2015 data run:

   1. API through which Firehose-picked and normalized clinical data are accessible has been renamed to Samples/Clinical_FH

   2. Verbatim TCGA clinical data may be accessed through the new Samples/Clincal API

   3. A CDE will be reflected in either API only when it has a value other than NA for at least 1 patient case in any disease cohort.

   4. For backward compatibility, the Samples/ClinicalTier1 remains available (as a synonym for Samples/Clinical_FH)
   5. fbget Python and UNIX CLI bindings have been suitably updated
   6. Which makes it extremely easy to determine for what patients and cohorts any given CDE is defined: e.g.
      fbget clinical cde=gleason_score
      will show each patient case with a valid gleason_score, across all cohorts
2. viewGene: now enforces rendering of at most 1 gene per Submit; if multiple genes are given the first is selected
3. iCoMut:
   1. Popup tooltips for mutation panels now show total # of mutations AND fractional % of a given type (e.g. missense)
   2. New search feature, enabling one to see into which clusters/panels/etc a given patient (or set of patients) falls
      
4. After extensive testing, upgraded backend database from Mongo2.x to Mongo3.x: v3 dramatically reduces the memory footprint and data storage sizes, which yields greater performance and also clears considerable breathing room to add more data APIs (e.g. for methylation, RPPA, etc) as well as AWG-specific databases

1. RPPA issues due to changes in new data file row names: these have been previously reported to MDACC.
   1. KIRP: ARHI-M-E -> DIRAS3-M-E (batch 2.0.0)
   2. LGG: Acetyl-a-Tubulin-Lys40-R-C -> Acetyl-a-Tubulin(Lys40)-R-C (batch 2.0.0)
   3. PAAD: DIRAS3-M-E -> ARHI-M-E (batch 1.2.0)
   4. STES: ARHI-M-E -> DIRAS3-M-E (batch 2.0.0)
2. RPPA issue with KIRC antibody file:

   1. Gene names were missing for 15 antibodies. The gene names were located in an online supplement file, and manually added:

      CA9	CA9
      SDHB	Complex-II_subunit30
      GYG1	GYG-Glycogenin1
      GYS1	GYS
      GYS1	GYS_pS641
      HIF1A	HIF-1_alpha
      LDHA	LDHA
      LDHB	LDHB
      MTCO2	Mitochondria
      ATP5A1	Oxphos-complex-V_subunitb
      PKM2	PKM2
      PYGB	PYGB
      PYGB	PYGB-AB2
      PYGL	PYGL
      PYGM	PYGM