The commands scc-search and scc-search-download can be used to search and download
“old” processing results, without having to provide the measurement IDs yourself. You can use
scc-search to see what is available and then use scc-search-download to download
files of interest.
pollyxt_pipelines scc-search [--location [<...>]] [--to-csv <...>] <date-start> <date-end>
date-start required: First day to search (inclusive)
date-end required: Last day to search (exclusive)
--location=: Optionally filter by a measurement station
--detailed-status: Fetch processing status codes (eg. 127) for each product. This option must be used with --to-csv=.
--to-csv=: Optionally store the results in a CSV file. This file can also be used with scc-download
This command should output a table with all measurements for the given time period and the available products. When using
--detailed-status, the resulting CSV will contain the processing status codes for each product. This option requires at least N
request for N measurements so it should be avoided if not required.
Sample CSV file (without --detailed-status):
id,station_id,location,date_start,date_end,date_creation,date_updated,hirelpp,cloudmask,elpp,elda,eldec,elic,elquick,is_processing
20210307aky0930,Antikythera,aky,2021-03-07T09:30:00,2021-03-07T10:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,OK,OK,OK,ERROR,NO_RUN,NO_RUN,OK,OK
20210307aky1030,Antikythera,aky,2021-03-07T10:30:00,2021-03-07T11:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,OK,OK,OK,ERROR,NO_RUN,NO_RUN,OK,OK
20210307aky1130,Antikythera,aky,2021-03-07T11:30:00,2021-03-07T12:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,OK,OK,OK,ERROR,NO_RUN,NO_RUN,OK,OK
Sample CSV file (with --detailed-status):
station_id,location,date_start,date_end,date_creation,date_updated,upload,hirelpp,cloudmask,elpp,elic
20210307aky0930,Antikythera,aky,2021-03-07T09:30:00,2021-03-07T10:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,127,127,127,-127,127
20210307aky1030,Antikythera,aky,2021-03-07T10:30:00,2021-03-07T11:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,127,127,127,-127,127
20210307aky1130,Antikythera,aky,2021-03-07T11:30:00,2021-03-07T12:30:00,2021-03-07T13:46:00,2021-03-07T13:46:00,127,127,127,-127,127
pollyxt_pipelines scc-search-download [--location [<...>]] <date-start> <date-end>
date-start required: First day to search (inclusive)
date-end required: Last day to search (exclusive)
--location=: Optionally filter by a measurement station
--no-hirelpp: Do not download HiRELPP products
--no-cloudmask: Do not download cloudmask products
--no-elpp: Do not download ELPP files
--no-optical: Do not download optical (ELDA or ELDEC) files
--no-elic: Do not download ELIC files
By default, this command downloads all products but you can specifically excluse some using the
--no-* options.
The lidar constants table can be downloaded as a CSV file using the lidar-constants-scc command.
pollyxt_pipelines scc-lidar-constants [--location [<...>]] <date-start> <date-end> <csv>
date-start required: First day to search (inclusive)
date-end required: Last day to search (exclusive)
csv required: Where to save the CSV file.
--location=: Optionally filter by a measurement station
Search for all files between 2020-01-01 and 2020-02-01 from Finokalia.
pollyxt_pipelines scc-search 2020-01-01 2020-02-01 --location=Finokalia
Download all files between 2020-10-07 and 2020-10-10 from all stations
pollyxt_pipelines scc-search-download 2020-10-07 2020-10-10
Download all files, excluding optical products, between 2020-10-07 and 2020-10-10 from Antikythera:
pollyxt_pipelines scc-search-download 2020-10-07 2020-10-10 --no-optical --location=Antikythera
Download the Lidar constant table for a given date range.
pollyxt_pipelines scc-lidar-constants 2020-10-01 2020-11-01 lidar.csv
All network-related operationg are handled by the pollyxt_pipelines.scc_access module, which
has a SCC class. For more information read the Network API page.