Page Comparison

dryrun

TRUE if you are just testing (will still show output)
FALSE if you want to push files to the metrics service.

url

This is the actual call to the metrics webservice that should be configured with your destination table and any “outer columns” (columns not being parsed out of your files, such as filename)

(More documentation TBD)

filepathRegex (optional; defaults to * if left blank)

Regex to pick which files to push (and also can extract data from filenames in capturing groups)

e.g. (.*)/(.*AspAllOutputQC.csv) or

(.*)/([^/]*)_tableau_metrics.txt

where the starting folder is specified in the delta parameter (below).
??? Is a regex needed for files that are listed individually (like if using FILES_CSV)?
Capturing groups need to be specified in OUTER COLUMNS in the url parameter.

delta

tells the service where to look for files

Use FILES_CSV to manually list files

FILES_CSV /home/unix/analytics/SK-3PJO_tableau_metrics.txt,XYZ-File.txt

Use FILES IN FOLDER to specify a folder location (your regular expression above will target files more specifically)

FILES IN FOLDER /seq/tableau_files/VVPVolumeQC

??? are there other existing options here besides FILES_CSV and FILES IN FOLDER?

TIMESTAMPED defines a relative or absolute time range (optional)

push files modified in the last 10 hours:

FILES IN FOLDER /home/unix/analytics TIMESTAMPED BETWEEN -10h AND NOW

push files modified after 2019-10-19:

FILES IN FOLDER /home/unix/analytics TIMESTAMPED BETWEEN 2019-10-19 13:11:46 AND NOW'

push files modified between the previous filepusher run and now. This requires a helper txt file to be created (with an initial timestamp), and file pusher will update it with the current time when run. It then serves as the start delta for the next time the file pusher is run.

FILES IN FOLDER /home/unix/analytics TIMESTAMPED BETWEEN /home/unix/analytics/mytimestamp.txt AND NOW??? does the timestamp helper need to be in the same location as the pushed files?