MSNoise API

msnoise.api.get_logger(name, loglevel=None, with_pid=False)

Returns the current configured logger or configure a new one.

msnoise.api.get_engine(inifile=None)

Returns the a SQLAlchemy Engine

Parameters

inifile (str) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini

Return type

sqlalchemy.engine.Engine

Returns

An Engine Object

msnoise.api.connect(inifile=None)

Establishes a connection to the database and returns a Session object.

Parameters

inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini

Return type

sqlalchemy.orm.session.Session

Returns

A Session object, needed for many of the other API methods.

msnoise.api.create_database_inifile(tech, hostname, database, username, password, prefix='')

Creates the db.ini file based on supplied parameters.

Parameters
  • tech (int) – The database technology used: 1=sqlite 2=mysql

  • hostname (string) – The hostname of the server (if tech=2) or the name of the sqlite file if tech=1)

  • database (string) – The database name

  • username (string) – The user name

  • prefix (string) – The prefix to use for all tables

  • password (string) – The password of user

Returns

None

msnoise.api.read_db_inifile(inifile=None)

Reads the parameters from the db.ini file.

Parameters

inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini

Return type

tuple

Returns

tech, hostname, database, username, password

msnoise.api.get_config(session, name=None, isbool=False, plugin=None)

Get the value of one or all config bits from the database.

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • name (str) – The name of the config bit to get. If omitted, a dictionnary with all config items will be returned

  • isbool (bool) – if True, returns True/False for config name. Defaults to False

  • plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.

Return type

str, bool or dict

Returns

the value for name or a dict of all config values

msnoise.api.update_config(session, name, value, plugin=None)

Update one config bit in the database.

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • name (str) – The name of the config bit to set.

  • value (str) – The value of parameter name. Can also be NULL if you don’t want to use this particular parameter.

  • plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.

msnoise.api.get_params(session)

Get config parameters from the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Returns

a Param class containing the parameters

msnoise.api.get_filters(session, all=False)

Get Filters from the database.

Parameters
Return type

list of Filter

Returns

a list of Filter

msnoise.api.update_filter(session, ref, low, mwcs_low, high, mwcs_high, rms_threshold, mwcs_wlen, mwcs_step, used)

Updates or Insert a new Filter in the database.

See also

msnoise.msnoise_table_def.declare_tables.Filter

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • ref (int) – The id of the Filter in the database

  • low (float) – The lower frequency bound of the Whiten function (in Hz)

  • high (float) – The upper frequency bound of the Whiten function (in Hz)

  • rms_threshold (float) – Not used anymore

  • mwcs_wlen (float) – Window length (in seconds) to perform MWCS

  • mwcs_step (float) – Step (in seconds) of the windowing procedure in MWCS

  • used (bool) – Is the filter activated for the processing

msnoise.api.get_networks(session, all=False)

Get Networks from the database.

Parameters
Return type

list of str

Returns

a list of network codes

msnoise.api.get_stations(session, all=False, net=None)

Get Stations from the database.

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • all (bool) – Returns all stations from the database if True, or only stations where used = 1 if False (default)

  • net (str) – if set, limits the stations returned to this network

Return type

list of msnoise.msnoise_table_def.declare_tables.Station

Returns

list of Station

msnoise.api.get_station(session, net, sta)

Get one Station from the database.

Parameters
Return type

msnoise.msnoise_table_def.declare_tables.Station

Returns

a Station Object

msnoise.api.update_station(session, net, sta, X, Y, altitude, coordinates='UTM', instrument='N/A', used=1)

Updates or Insert a new Station in the database.

See also

msnoise.msnoise_table_def.declare_tables.Station

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • net (str) – The network code of the Station

  • sta (str) – The station code

  • X (float) – The X coordinate of the station

  • Y (float) – The Y coordinate of the station

  • altitude (float) – The altitude of the station

  • coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.

  • instrument (str) – The instrument code, useful with PAZ correction

  • used (bool) – Whether this station must be used in the computations.

msnoise.api.get_station_pairs(session, used=None, net=None)

Returns an iterator over all possible station pairs. If auto-correlation is configured in the database, returns N*N pairs, otherwise returns N*(N-1)/2 pairs.

Parameters
Return type

iterable

Returns

An iterable of Station object pairs

msnoise.api.get_interstation_distance(station1, station2, coordinates='DEG')

Returns the distance in km between station1 and station2.

Warning

Currently the stations coordinates system have to be the same!

Parameters
  • station1 (Station) – A Station object

  • station2 (Station) – A Station object

  • coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.

Return type

float

Returns

The interstation distance in km

msnoise.api.update_data_availability(session, net, sta, comp, path, file, starttime, endtime, data_duration, gaps_duration, samplerate)

Updates a DataAvailability object in the database

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • net (str) – The network code of the Station

  • sta (str) – The station code

  • comp (str) – The component (channel)

  • path (str) – The full path to the folder containing the file

  • file (str) – The name of the file

  • starttime (datetime.datetime) – Start time of the file

  • endtime (datetime.datetime) – End time of the file

  • data_duration (float) – Cumulative duration of available data in the file

  • gaps_duration (float) – Cumulative duration of gaps in the file

  • samplerate (float) – Sample rate of the data in the file (in Hz)

msnoise.api.get_new_files(session)

Returns the files marked “N”ew or “M”odified in the database

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

list

Returns

list of DataAvailability

msnoise.api.get_data_availability(session, net=None, sta=None, comp=None, starttime=None, endtime=None)

Returns the DataAvailability objects for specific net, sta, starttime or endtime

Parameters
Return type

list

Returns

list of DataAvailability

msnoise.api.mark_data_availability(session, net, sta, flag)

Updates the flag of all DataAvailability objects matching net.sta in the database

Parameters
msnoise.api.count_data_availability_flags(session)

Count the number of DataAvailability, grouped by flag

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

list

Returns

list of [count, flag] pairs

msnoise.api.update_job(session, day, pair, jobtype, flag, commit=True, returnjob=True, ref=None)

Updates or Inserts a new Job in the database.

Parameters
  • day (str) – The day in YYYY-MM-DD format

  • pair (str) – the name of the pair (EXAMPLE?)

  • jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?

  • flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.

  • commit (bool) – Whether to directly commit (True, default) or not (False)

  • returnjob (bool) – Return the modified/inserted Job (True, default) or not (False)

Return type

Job or None

Returns

If returnjob is True, returns the modified/inserted Job.

msnoise.api.massive_insert_job(jobs)

Routine to use a low level function to insert much faster a list of Job. This method uses the Engine directly, no need to pass a Session object.

Parameters

jobs (list) – a list of Job to insert.

msnoise.api.massive_update_job(session, jobs, flag='D')

Routine to use a low level function to update much faster a list of Job. This method uses the Job.ref which is unique.

Parameters
  • jobs (list) – a list of Job to update.

  • flag (str) – The destination flag.

msnoise.api.is_next_job(session, flag='T', jobtype='CC')

Are there any Job in the database, with flag=`flag` and jobtype=`type`

Parameters
Return type

bool

Returns

True if at least one Job matches, False otherwise.

msnoise.api.get_next_job(session, flag='T', jobtype='CC')

Get the next Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs of the same type are grouped per day. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters
Return type

list

Returns

list of Job

msnoise.api.is_dtt_next_job(session, flag='T', jobtype='DTT', ref=False)

Are there any DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. If ref is provided, checks if a DTT “REF” job is present.

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?

  • flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.

  • ref (bool) – Whether to check for a REF job (True) or not (False, default)

Return type

bool

Returns

True if at least one Job matches, False otherwise.

msnoise.api.get_dtt_next_job(session, flag='T', jobtype='DTT')

Get the next DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs are then grouped per station pair. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters
Return type

tuple

Returns

(pairs, days, refs): List of station pair names - Days of the next DTT jobs - Job IDs (for later being able to update their flag).

msnoise.api.reset_jobs(session, jobtype, alljobs=False, rule=None)

Sets the flag of all jobtype Jobs to “T”odo.

Parameters
msnoise.api.reset_dtt_jobs(session, pair)

Sets the flag of all DTT Jobs of one pair to “T”odo.

Parameters
msnoise.api.get_job_types(session, jobtype='CC')

Count the number of Jobs of a specific type, grouped by flag.

Parameters
Return type

list

Returns

list of [count, flag] pairs

msnoise.api.get_jobs_by_lastmod(session, jobtype='CC', lastmod=datetime.datetime(2020, 2, 25, 17, 25, 15, 593597))
Parameters
Return type

list

Returns

list of Job objects.

msnoise.api.export_allcorr(session, ccfid, data)
msnoise.api.export_allcorr2(session, ccfid, data)
msnoise.api.add_corr(session, station1, station2, filterid, date, time, duration, components, CF, sampling_rate, day=False, ncorr=0, params=None)

Adds a CCF to the data archive on disk.

Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • station1 (str) – The name of station 1 (formatted NET.STA)

  • station2 (str) – The name of station 2 (formatted NET.STA)

  • filterid (int) – The ID (ref) of the filter

  • date (datetime.date or str) – The date of the CCF

  • time (datetime.time or str) – The time of the CCF

  • duration (float) – The total duration of the exported CCF

  • components (str) – The name of the components used (ZZ, ZR, …)

  • sampling_rate (float) – The sampling rate of the exported CCF

  • day (bool) – Whether this function is called to export a daily stack (True) or each CCF (when keep_all parameter is set to True in the configuration). Defaults to True.

  • ncorr (int) – Number of CCF that have been stacked for this CCF.

  • params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().

msnoise.api.export_sac(db, filename, pair, components, filterid, corr, ncorr=0, sac_format=None, maxlag=None, cc_sampling_rate=None, params=None)
msnoise.api.export_mseed(db, filename, pair, components, filterid, corr, ncorr=0, maxlag=None, cc_sampling_rate=None, params=None)
msnoise.api.stack(data, stack_method='linear', pws_timegate=10.0, pws_power=2, goal_sampling_rate=20.0)
Parameters
  • data (numpy.ndarray) – the data to stack, each row being one CCF

  • stack_method (str) – either linear: average of all CCF or pws to compute the phase weigthed stack. If pws is selected, the function expects the pws_timegate and pws_power.

  • pws_timegate (float) – PWS time gate in seconds. Width of the smoothing window to convolve with the PWS spectrum.

  • pws_power (float) – Power of the PWS weights to be applied to the CCF stack.

  • goal_sampling_rate (float) – Sampling rate of the CCF array submitted

Return type

numpy.array

Returns

the stacked CCF.

msnoise.api.get_results(session, station1, station2, filterid, components, dates, mov_stack=1, format='stack', params=None)
Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • station1 (str) – The name of station 1 (formatted NET_STA)

  • station2 (str) – The name of station 2 (formatted NET_STA)

  • filterid (int) – The ID (ref) of the filter

  • components (str) – The name of the components used (ZZ, ZR, …)

  • dates (list) – List of TODO datetime.datetime

  • mov_stack (int) – Moving window stack.

  • format (str) – Either stack: the data will be stacked according to the parameters passed with params or matrix: to get a 2D array of CCF.

  • params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().

Return type

numpy.ndarray

Returns

Either a 1D CCF (if format is stack or a 2D array (if format= matrix).

msnoise.api.get_results_all(session, station1, station2, filterid, components, dates)
Parameters
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

  • station1 (str) – The name of station 1 (formatted NET_STA)

  • station2 (str) – The name of station 2 (formatted NET_STA)

  • filterid (int) – The ID (ref) of the filter

  • components (str) – The name of the components used (ZZ, ZR, …)

  • dates (list) – List of TODO datetime.datetime

Return type

pandas.DataFrame

Returns

All CCF results in a pandas.DataFrame, where the index is the time of the CCF and the columns are the times in the coda.

msnoise.api.get_maxlag_samples(session)

Returns the length of the CC functions. Gets the maxlag and sampling rate from the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

int

Returns

the length of the CCF in samples

msnoise.api.get_t_axis(session)

Returns the time axis (in seconds) of the CC functions. Gets the maxlag from the database and uses get_maxlag_samples function.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

numpy.array

Returns

the time axis in seconds

msnoise.api.get_components_to_compute(session, plugin=None)

Returns the components configured in the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

list of str

Returns

a list of components to compute

msnoise.api.get_components_to_compute_single_station(session, plugin=None)

Returns the components configured in the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

list of str

Returns

a list of components to compute

msnoise.api.build_ref_datelist(session)

Creates a date array for the REF. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

tuple

Returns

(start, end, datelist)

msnoise.api.build_movstack_datelist(session)

Creates a date array for the analyse period. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

tuple

Returns

(start, end, datelist)

msnoise.api.updated_days_for_dates(session, date1, date2, pair, jobtype='CC', interval=datetime.timedelta(days=1), returndays=False)

Determines if any Job of jobtype=`jobtype` and for pair=`pair`, concerning a date between date1 and date2 has been modified in the last interval=`interval`.

Parameters
Return type

list or bool

Returns

List of days if returndays is True, only “True” if not. (not clear!)

msnoise.api.azimuth(coordinates, x0, y0, x1, y1)

Returns the azimuth between two coordinate sets.

Parameters
  • coordinates (str) – {‘DEG’, ‘UTM’, ‘MIX’}

  • x0 (float) – X coordinate of station 1

  • y0 (float) – Y coordinate of station 1

  • x1 (float) – X coordinate of station 2

  • y1 (float) – Y coordinate of station 2

Return type

float

Returns

The azimuth in degrees

msnoise.api.nextpow2(x)

Returns the next power of 2 of x.

Parameters

x (int) – any value

Return type

int

Returns

the next power of 2 of x

msnoise.api.check_and_phase_shift(trace, taper_length=20.0)
msnoise.api.getGaps(stream, min_gap=None, max_gap=None)
msnoise.api.make_same_length(st)

This function takes a stream of equal sampling rate and makes sure that all channels have the same length and the same gaps.

msnoise.api.clean_scipy_cache()

This functions wraps all destroy scipy cache at once. It is a workaround to the memory leak induced by the “caching” functions in scipy fft.

msnoise.api.preload_instrument_responses(session)

This function preloads all instrument responses from response_format and stores the seed ids, start and end dates, and paz for every channel in a DataFrame.

Warning

This function only works for response_format being “inventory” or “dataless”.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()

Return type

pandas.DataFrame

Returns

A table containing all channels with the time of operation and poles and zeros.