/
Jira Inspector / our "discovery" tool

Jira Inspector / our "discovery" tool

Owned by Atanas Mihalev
Last updated: 2023-08-09
8 min read

Motivation

Over the years our LabOpsJira has been extended with a multitude of projects, workflows, issue-types, custom-fields, etc. Regardless of where our new Jira home would be (Atlassian Cloud or DataCenter), at some point we will have to conduct a User Acceptance Tests (UAT) to make sure that crucial functionality is not affected. These UATs would be very tightly linked to a specific project/workflow (for example: LCSET tests would look very differently from PDO tests). Given the variety of projects, collective knowledge is mostly stored in people’s minds. So just brainstorming with multiple teams about “what the current projects/workflows are and who is doing what” will not be very productive.
We need a deep Analytics off Jira itself which could provide a complete picture of active project diagrams, issue-types, workflows, users, transitions, LIMS interactions and more.

Having done such discovery upfront, 3POs

  • would be able to target the most important projects/workflows 1st (resp. prioritize over time),

  • would know which team to contact (along with specific names)

  • would have a project/workflow diagram printout at hand along with specific issue-examples to drive the discussion

Discussions with LOJ teams would become a data-driven workshop making sure that all bases are covered.

ProjectWorkflows - see all workflows for a given project

https://analytics.broadinstitute.org/JiraInspector/ProjectWorkflows?project=LCSET&workflow_name=show&issue_type=show&actual_status_coverage=hide&days_to_look_back=180&jira=labopsjira&format=html

  • how massive a project/workflow is (N_issues)

  • how dependent a project/workflow is on apps (N_REFS, N_DREFS)

  • is it current or outdated project/workflow (Latest Creation Date)

  • print out the generic workflow diagram

  • get representative issue tickets to drill further via IssueStepsTimeMachine tool

  • find the “driven-by-Mercury” Jira actions (all steps where username='squid')

  • use these params to show/hide/filter a specific field:

    • issue_type=show | hide

    • app=show | hide

    • app_widget=show | hide

      • post-functions, validators, actions, etc - magical bits of automation triggered implicitly behind the scene (send email, update custom field, create a new issue in another project, etc)

    • app_filter=all | atlassian | non-atlassian

      • “atlassian” app_widgets are believed to be easy to migrate since Atlassian is behind it

      • “non-atlassian” app_widgets are believed to be harder to migrate since 3rd party smaller vendors are behind it

      • filtered-apps N_REFS are still rolled up even if app=hide param is applied

    • wf_component_type=show | hide

      • in addition to “steps” a workflow could also have post-functions attached to “global-actions”, “common-actions” or “initial-actions” - all these are collectively called “workflow_components“ (or wf_components)

    • wf_component_type_filter=all | step-actions | global-actions | common-actions | initial-actions

      • filtered wf_omponent_types N_REFS are still rolled up even if wf_omponent_types=hide param is applied

    • how far in the past to look: days_to_look_back=4000

    • which Jira instance: jira=labopsjira | gpinfojira

    • project to look into: project=all | decisions_only | LCSET

    • do you want to look into the browser or download to CSV: format=html | csv | tsv

    • actual_status_coverage=show | hide

      • group issues by actual statuses a ticket has gone through, it will also produce all users involved

IssueStepsTimeMachine - look at all the steps a given ticket has gone through. Nice start for drafting a UAT (user acceptance test)

https://analytics.broadinstitute.org/JiraInspector/IssueStepsTimeMachine?issue=LCSET-27251&jira=labopsjira&format=html

ProjectsIssuetypesFields - look at all CustomFields along with apps supporting them

ProjectWorkflowsAppDependencies across both instances - in terms of N_REFs (number of app-references in the workflow descriptor)

https://analytics.broadinstitute.org/JiraInspector/ProjectWorkflowsAppDependencies

ActiveUsers - users who have logged-in or created in the past N days

https://analytics.broadinstitute.org/JiraInspector/ActiveUsers?jira=labopsjira&format=html&days_to_look_back=180

  • how far in the past to look: days_to_look_back=180

  • which Jira instance: jira=labopsjira | gpinfojira

  • format=html | csv

ProjectLinksOverview - how projects are interlinked (last 30days worth of links)

https://analytics.broadinstitute.org/JiraInspector/ProjectLinksOverview?project=LCSET&jira=labopsjira

  • which Jira instance: jira=labopsjira | gpinfojira

  • project to look into: project=LCSET (all also works)

Plugins - all plugins directory (in the most cases stay away from this one)

https://analytics.broadinstitute.org/JiraInspector/Plugins

Examples of gradually drilling top-down from Projects, Workflows, WfComponents, Apps, AppWidgets

Technical dissection of the Workflow descriptor XML

All the ProjectWorkflow information is derived from the so called Worlfkow descriptor XML recorded in Jira database. While some of this information is exposed in Jira’s Workflow pages, there is also a lot of information (like app dependencies) that is not visible in these pages. Also, even for the info available it would take dozens of clicking on multiple confusing pages to get to the info you need. So, a “bottom-up“ approach seemed an interesting option where we extract all the XML descriptors, parse out all the data, flatten it out into a table and make it available for further SQL-supported discovery JOINing with the rest of our discovery-datasets.
Don’t worry - even if just a small percentage of this dissection sticks with you it would still help you a lot when leveraging the JiraInsector “discovery“ tools.
analytics.broadinstitute.org/JiraInspector/WorkflowDescriptor?jira=labopsjira&workflow_name=LCSET Workflow with Plating
Let’s dive into a workflow descriptor XML to see how this works:

Workflow Components break down
post-functions exposed along with apps behind it

 

Every place in the Workflow XML where a post-function (or validator or condition) is mentioned we call it an “app-reference”, or just “reference”. While decoding all the references might seem like a daunting task, a lot can be discovered just by counting the “number of references” (N_REFS) rolledup by APPS, by Projects, etc. Every “reference” is a potential drag on the migration.
”Migrating a project” boils down to migrating over:
-- all the “workflow widgets” (post-functions, validators, conditions, etc) for that project, so the greater the N_REFS is the more difficult it is.
-- all the “custom fields” for that project
Another interesting metrics is the number of apps those N_REFS are distributed over. (for example “10 N_REFs supported by 3 apps” might be more difficult than “20 N_REFS supported by only 1 APP”)

How N_REFS can be leveraged:

  1. How difficult to migrate my project is? (could be read as “what are the chances of successful migration”?)

  2. What is the N_REFS number for it?

  3. Is the N_REFS distributed across multiple APPS? Which APPS?

  4. Are these APPS marked by JCMA as “Stage1”, “Stage2” or “Not supported in Cloud“?

  5. What remedies might be available for these features (app-widgets) that are not supported in Cloud?

  6. If it turns out it’s very difficult and/or expensive, shall I contact my stakeholders and discuss if we could give up that feature?

 

The almighty “post-functions”

Q: Why do we care about post-functions?
A: post-functions are an essential part of what a “working Jira project” means. These are all the automations being triggered behind the scene for you when a ticket is moving through the steps in the workflow. (for example: send email, create a new ticket in another project, post a comment, parse a field and update another field, etc)

Q: Our UATs already test the workflow “transitions” - why do we have to deal with “post-functions”?
A: Without covering “post-functions” (resp. transition automations), an UAT is worthless.

Q: post-functions are difficult and time-consuming to deal with, why don’t we just wait for consultants to provide “insights” before we tackle these?
A: No amount of consultant insights can eliminate the core business need for us knowing our own automation rules.
Writing good-coverage UATs, executing UATs is all on us/GP.
UATs must be executed on all projects***, all workflows, all transitions along with all automations off the transitions (resp. post-functions).
*** only projects decided to be migrated are referenced here

Q: How to find the list of post-functions for a given workflow/transition?
A: The “3-clicks post-function lookup” procedure (snapshot below) is the only method we got so far.
You start off with the workflow viewer.
https://labopsjira.broadinstitute.org/secure/admin/workflows/ViewWorkflowSteps.jspa?workflowMode=live&workflowName=Whole+Genome+LCSET+v2&descriptorTab=postfunctions

Q: What’s the point of these APP_WIDGETS ?
A: In addition to “post-functions”, there are few more less famous workflow automations: conditions, validators and triggers. (example below) Collectively we call all these APP_WIDGETs.

 

 

Storytelling with Project/Workflow and IssueStepsTimeMachine

 

Caveats

  • Jira Inspector leverages direct access to the Oracle DB underneath, similar implementation in Cloud is not possible