mirar.pipelines package

Subpackages

• mirar.pipelines.sedmv2 package
  • Subpackages
    • mirar.pipelines.sedmv2.config package
      • Submodules
      • mirar.pipelines.sedmv2.config.constants module
  • Submodules
  • mirar.pipelines.sedmv2.blocks module
  • mirar.pipelines.sedmv2.generator module
    • sedmv2_astrometric_catalog_generator()
    • sedmv2_photometric_catalog_generator()
    • sedmv2_reference_image_generator()
    • sedmv2_reference_image_resampler()
    • sedmv2_reference_psfex()
    • sedmv2_reference_sextractor()
    • sedmv2_zogy_catalogs_purifier()
  • mirar.pipelines.sedmv2.load_sedmv2_image module
    • clean_cal_header()
    • clean_science_header()
    • date_obs_to_mjd()
    • load_raw_sedmv2_mef()
    • load_sedmv2_mef_image()
  • mirar.pipelines.sedmv2.sedmv2_pipeline module
    • SEDMv2Pipeline
      • SEDMv2Pipeline.all_pipeline_configurations
      • SEDMv2Pipeline.default_cal_requirements
      • SEDMv2Pipeline.download_raw_images_for_night()
      • SEDMv2Pipeline.name
      • SEDMv2Pipeline.non_linear_level
• mirar.pipelines.summer package
  • Subpackages
    • mirar.pipelines.summer.config package
      • Subpackages
      • Submodules
      • mirar.pipelines.summer.config.constants module
    • mirar.pipelines.summer.models package
      • Submodules
      • mirar.pipelines.summer.models.base_model module
  • Submodules
  • mirar.pipelines.summer.blocks module
  • mirar.pipelines.summer.generator module
    • summer_astrometric_catalog_generator()
    • summer_photometric_catalog_generator()
    • summer_photometric_img_catalog_purifier()
    • summer_reference_image_generator()
    • summer_reference_image_resampler()
    • summer_reference_psfex()
    • summer_reference_sextractor()
    • summer_zogy_catalogs_purifier()
  • mirar.pipelines.summer.load_summer_image module
    • load_proc_summer_image()
    • load_raw_summer_fits()
    • load_raw_summer_image()
  • mirar.pipelines.summer.summer_pipeline module
    • SummerPipeline
      • SummerPipeline.all_pipeline_configurations
      • SummerPipeline.default_cal_requirements
      • SummerPipeline.download_raw_images_for_night()
      • SummerPipeline.name
• mirar.pipelines.winter package
  • Subpackages
    • mirar.pipelines.winter.config package
    • mirar.pipelines.winter.models package
      • set_up_q3c()
      • Submodules
      • mirar.pipelines.winter.models.base_model module
  • Submodules
  • mirar.pipelines.winter.blocks module
  • mirar.pipelines.winter.build_references module
    • dummy_split_image_batch_generator()
    • run_winter_reference_build_pipeline()
    • write_subfields_file()
  • mirar.pipelines.winter.constants module
  • mirar.pipelines.winter.fix_headers module
    • fix_headers()
  • mirar.pipelines.winter.generator module
    • ReductionQualityError
    • check_winter_local_catalog_overlap()
    • ref_sextractor()
    • select_winter_flat_images()
    • winter_astrometric_ref_catalog_generator()
    • winter_astrometric_ref_catalog_namer()
    • winter_astrometry_sextractor_catalog_purifier()
    • winter_astrostat_catalog_purifier()
    • winter_candidate_annotator_filterer()
    • winter_candidate_avro_fields_calculator()
    • winter_candidate_quality_filterer()
    • winter_fourier_filtered_image_generator()
    • winter_photometric_catalog_generator()
    • winter_photometric_ref_catalog_namer()
    • winter_ref_catalog_namer()
    • winter_ref_photometric_img_catalog_purifier()
    • winter_reference_generator()
    • winter_reference_image_resampler_for_zogy()
    • winter_reference_phot_calibrator()
    • winter_reference_psfex()
    • winter_reference_sextractor()
    • winter_reference_stack_annotator()
    • winter_stackid_annotator()
    • winter_wfau_component_image_stacker()
  • mirar.pipelines.winter.load_winter_image module
    • annotate_winter_subdet_headers()
    • clean_header()
    • get_raw_winter_mask()
    • load_raw_winter_mef()
    • load_stacked_winter_image()
    • load_test_winter_image()
    • load_winter_mef_image()
    • load_winter_stack()
  • mirar.pipelines.winter.winter_pipeline module
    • WINTERPipeline
      • WINTERPipeline.all_pipeline_configurations
      • WINTERPipeline.default_cal_requirements
      • WINTERPipeline.download_raw_images_for_night()
      • WINTERPipeline.name
      • WINTERPipeline.non_linear_level
• mirar.pipelines.wirc package
  • Subpackages
    • mirar.pipelines.wirc.wirc_files package
  • Submodules
  • mirar.pipelines.wirc.blocks module
  • mirar.pipelines.wirc.generator module
    • wirc_astrometric_catalog_generator()
    • wirc_photometric_catalog_generator()
    • wirc_photometric_img_catalog_purifier()
    • wirc_reference_image_generator()
    • wirc_reference_image_resampler()
    • wirc_reference_psfex()
    • wirc_reference_sextractor()
    • wirc_source_table_filter_annotator()
    • wirc_zogy_catalogs_purifier()
  • mirar.pipelines.wirc.load_wirc_image module
    • load_raw_wirc_fits()
    • load_raw_wirc_image()
  • mirar.pipelines.wirc.wirc_pipeline module
    • WircPipeline
      • WircPipeline.all_pipeline_configurations
      • WircPipeline.download_raw_images_for_night()
      • WircPipeline.name
      • WircPipeline.non_linear_level

Submodules

mirar.pipelines.base_pipeline module

Module containing the base of the Pipeline

class.

Each Pipeline will have several configurations. A configuration corresponds to a list of BaseProcessor objects.

The pipeline will process data using a chosen list of these individual BaseProcessor objects.

class mirar.pipelines.base_pipeline.Pipeline(selected_configurations: str | list[str] = 'default', night: int | str = '')

Bases: object

Base class for pipelines.

Each pipeline must have the following class variables:

• a name (the name of the instrument

• pipeline configurations

• gain

• a _load_raw_image() function

to load raw images and modify the headers etc as required

add_configuration(configuration_name: str, configuration: list[BaseProcessor])

Add a new configuration to the pipeline.

Parameters:

• configuration_name – Name of new configuration

• configuration – the list of processors

Returns:

None

property all_pipeline_configurations

Dictionary containing all pipeline configurations

static configure_processors(processors: list[BaseProcessor], sub_dir: str = '') → list[BaseProcessor]

Propagates the correct nightly setting to a list of processors.

Parameters:

• processors – Processors to configure

• sub_dir – night sub directory to use

Returns:

Updated processors

default_cal_requirements = None

static download_raw_images_for_night(night: str | int)

Function to download images from a remote server

Parameters:

night – Night of data to download

Returns:

None

get_error_output_path() → Path

Generates a unique path for the error summary, in the output data directory. Makes the parent directory structure if needed.

Returns:

path for error summary

load_pipeline_configuration(configuration: str = 'default') → list[BaseProcessor]

Load a particular named configuration from self.all_pipeline_configurations

Parameters:

configuration – configuration to be used

Returns:

list of processors

load_raw_image(path: str) → ImageBatch

Function to load in a raw image and create an Image object which can then be processed further

Parameters:

path – path of raw image

Returns:

Image object

property name

Unique name of pipeline , used to call it from the command via get_pipeline(). Should be the name of the instrument.

property non_linear_level

Non-linear level of instrument

pipelines = {'sedmv2': <class 'mirar.pipelines.sedmv2.sedmv2_pipeline.SEDMv2Pipeline'>, 'summer': <class 'mirar.pipelines.summer.summer_pipeline.SummerPipeline'>, 'winter': <class 'mirar.pipelines.winter.winter_pipeline.WINTERPipeline'>, 'wirc': <class 'mirar.pipelines.wirc.wirc_pipeline.WircPipeline'>}

postprocess_configuration(errorstack: ErrorStack, selected_configurations: str | list[str], processed_images: list[str] | None = None) → list[BaseProcessor]

Generate a postprocessing/cleanup processor sequence, Used by Monitor class for realtime processing. The first step is to update the header of images with any saved errors in errorstack.

Parameters:

• errorstack – Caught errors

• selected_configurations – Configurations to use.

• processed_images – list of processed images

Returns:

list of postprocess processors

reduce_images(dataset: Dataset | None = None, output_error_path: str | None = None, catch_all_errors: bool = True, selected_configurations: str | list[str] | None = None) → tuple[Dataset, ErrorStack]

Function to process a given dataset.

Parameters:

• dataset – dataset to process (can be empty)

• output_error_path – optional path to write error summary

• catch_all_errors – Either catch errors, or just immediately raise them

• selected_configurations – Configuration to use

Returns:

Post-processing dataset and summary of errors caught

set_configuration(new_configuration: str = 'default') → list[BaseProcessor]

Loads a new configuration for the pipeline

Parameters:

new_configuration – name of configuration

Returns:

list of corresponding processors

unpack_raw_image(path: str) → Image | list[Image]

Function to load in a raw image and ensure it has

the correct format for the code.

This function, unlike _load_raw_image() is not protected.

Parameters:

path – path of raw image

Returns:

The image data and image header