Getting Started#
The following tutorial is for CLI users, if you are using the library to write your own Python or R scripts then you’ll want to read Developers.
Installation#
We recommend using a conda environment just for CTDFjorder. To do this, open your terminal (MacOS/Linux) or command prompt (Windows) and run the following commands:
$ conda deactivate
$ conda create --name ctdfjorder python=3.12
$ conda activate ctdfjorder
To use CTDFjorder, first install it using pip:
(ctdfjorder) $ pip install ctdfjorder
Run ctdcli#
Now we will process our files.
Tip
To see what options you have to process the files, type ctdcli default -h or view the documentation for the CLI.
For the purposes of this demo are assuming that you have the following:
Files with endings .rsk from an RBR instrument or .csv from a Castaway device.
- A master sheet which will be used to attach metadata to the CTD tables. This must be named mastersheet.csv and be located in the same folder as your CTD data. Additionally it must have the following fields:
UNIQUE ID CODE
nominal longitude
nominal latitude
CTD cast file name
location
loc id
date/time (ISO)
sechhi depth
Access to a public MapBox token.
If you meet those conditions make your terminal window fullscreen.
Then copy and paste the following into your terminal, and replace MY_TOKEN with your public MapBox token.
Tip
Are you a member of Fjord Phyto? If so run command this instead poetry run ctdcli fjord --mld-ref 20 30 --mld-delta 0.05 --token pk.eyJ1Ijoibmlrb3Rob21hcyIsImEiOiJjbHl2Z2JzbDQxZjEwMmpwd2c1cnJpYmRyIn0.j9l0EXWa2ik51AbAcIe5HQ
(ctdfjorder) $ ctdcli default -r -p -m "mastersheet.csv" -w 4 --show-status --token MY_TOKEN
Interpret output#
If you see a spinning globe you did it! Once the files are done processing a table will print with pipeline information for each file. Green means the file passed a step and red means an error occurred such that the file could not continue to be processed. Once all files are completed, a map will open as well. The points are individual casts. The map can be filtered.
Plots are in the ctdplots folder next to our original data and were made with functions from the Visualize module.
There you will also find a ctdfjorder_data.csv with our processed data.
To investigate files that did not pass the pipeline open the ctdfjorder.log file.
Congrats! You can now use CTDFjorder to investigate your ctd data. For more in depth information on the processes executed here, read the API Documentation.
CLI Commands#
CTDFjorder
usage: sample [-h] {default,fjord} ...
Positional Arguments#
- command
Possible choices: default, fjord
Sub-commands#
default#
Run the default processing pipeline
sample default [-h] [-p] [-v]
[--mld-ref {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25} [{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25} ...]]
[--mld-delta {0.01,0.02,0.03,0.04,0.05} [{0.01,0.02,0.03,0.04,0.05} ...]]
[-q] [-r] [-s] [-d] [-m MASTERSHEET] [-w [WORKERS]]
[--token TOKEN] [-o OUTPUT]
[--filter-columns [{timestamp,year,month,filename,chlorophyll,temperature,sea_pressure,depth,salinity,speed_of_sound,specific_conductivity,conductivity,pressure,salinity_abs,surface_salinity,surface_temperature,surface_density,meltwater_fraction,density,potential_density,brunt_vaisala_frequency_squared,p_mid,secchi_depth,latitude,longitude,unique_id,profile_id,site_id,site_name,oxygen_concentration,oxygen_saturation,nitrate,phosphate,silicate,ph,alkalinity,turbidity,particulate_organic_carbon,total_organic_carbon,particulate_inorganic_carbon,dissolved_inorganic_carbon,chlorophyll_fluorescence,par,ammonium,orp} ...]]
[--filter-upper [FILTER_UPPER ...]]
[--filter-lower [FILTER_LOWER ...]]
Named Arguments#
- -p, --plot
Generate plots
Default:
False- -v, --verbose
Verbose logger output to ctdfjorder.log (repeat for increased verbosity)
Default:
0- --mld-ref
Possible choices: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
Reference value(s) for mld calculation.
Default:
20- --mld-delta
Possible choices: 0.01, 0.02, 0.03, 0.04, 0.05
Delta value(s) for mld calculation.
Default:
0.05- -q, --quiet
Quiet output (show errors only)
Default:
0- -r, --reset
Reset file environment
Default:
False- -s, --show-status
Show processing status and pipeline status
Default:
False- -d, --debug-run
Run 20 files for testing
Default:
False- -m, --mastersheet
Path to mastersheet
- -w, --workers
Max workers
- --token
MapBox token to enable interactive map plot
- -o, --output
Output file path
Default:
'ctdfjorder_data.csv'- --filter-columns
Possible choices: timestamp, year, month, filename, chlorophyll, temperature, sea_pressure, depth, salinity, speed_of_sound, specific_conductivity, conductivity, pressure, salinity_abs, surface_salinity, surface_temperature, surface_density, meltwater_fraction, density, potential_density, brunt_vaisala_frequency_squared, p_mid, secchi_depth, latitude, longitude, unique_id, profile_id, site_id, site_name, oxygen_concentration, oxygen_saturation, nitrate, phosphate, silicate, ph, alkalinity, turbidity, particulate_organic_carbon, total_organic_carbon, particulate_inorganic_carbon, dissolved_inorganic_carbon, chlorophyll_fluorescence, par, ammonium, orp
List of columns to filter
- --filter-upper
Upper bounds for the filtered columns
- --filter-lower
Lower bounds for the filtered columns
fjord#
Run the Fjord Phyto processing pipeline
sample fjord [-h] [-p] [-v]
[--mld-ref {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25} [{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25} ...]]
[--mld-delta {0.01,0.02,0.03,0.04,0.05} [{0.01,0.02,0.03,0.04,0.05} ...]]
[-q] [-r] [-s] [-d] [-m MASTERSHEET] [-w [WORKERS]]
[--token TOKEN] [-o OUTPUT]
[--filter-columns [{timestamp,year,month,filename,chlorophyll,temperature,sea_pressure,depth,salinity,speed_of_sound,specific_conductivity,conductivity,pressure,salinity_abs,surface_salinity,surface_temperature,surface_density,meltwater_fraction,density,potential_density,brunt_vaisala_frequency_squared,p_mid,secchi_depth,latitude,longitude,unique_id,profile_id,site_id,site_name,oxygen_concentration,oxygen_saturation,nitrate,phosphate,silicate,ph,alkalinity,turbidity,particulate_organic_carbon,total_organic_carbon,particulate_inorganic_carbon,dissolved_inorganic_carbon,chlorophyll_fluorescence,par,ammonium,orp} ...]]
[--filter-upper [FILTER_UPPER ...]]
[--filter-lower [FILTER_LOWER ...]]
Named Arguments#
- -p, --plot
Generate plots
Default:
False- -v, --verbose
Verbose logger output to ctdfjorder.log (repeat for increased verbosity)
Default:
0- --mld-ref
Possible choices: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
Reference value(s) for mld calculation.
Default:
20- --mld-delta
Possible choices: 0.01, 0.02, 0.03, 0.04, 0.05
Delta value(s) for mld calculation.
Default:
0.05- -q, --quiet
Quiet output (show errors only)
Default:
0- -r, --reset
Reset file environment
Default:
False- -s, --show-status
Show processing status and pipeline status
Default:
False- -d, --debug-run
Run 20 files for testing
Default:
False- -m, --mastersheet
Path to mastersheet
- -w, --workers
Max workers
- --token
MapBox token to enable interactive map plot
- -o, --output
Output file path
Default:
'ctdfjorder_data.csv'- --filter-columns
Possible choices: timestamp, year, month, filename, chlorophyll, temperature, sea_pressure, depth, salinity, speed_of_sound, specific_conductivity, conductivity, pressure, salinity_abs, surface_salinity, surface_temperature, surface_density, meltwater_fraction, density, potential_density, brunt_vaisala_frequency_squared, p_mid, secchi_depth, latitude, longitude, unique_id, profile_id, site_id, site_name, oxygen_concentration, oxygen_saturation, nitrate, phosphate, silicate, ph, alkalinity, turbidity, particulate_organic_carbon, total_organic_carbon, particulate_inorganic_carbon, dissolved_inorganic_carbon, chlorophyll_fluorescence, par, ammonium, orp
List of columns to filter
- --filter-upper
Upper bounds for the filtered columns
- --filter-lower
Lower bounds for the filtered columns