Advanced usage tutorial¶

This notebook assumes that you are already familiar with the hiresprv Python package and have run through the process described in the basic data reduction tutorial notebook.

This notebook is meant to be a template to set up your own processing using the code snippets provided here. The notebook may not run exactly as presented if you attempt to run the full notebook directly. The code has not been rigorously tested from within a notebook environemnt.

The notebook is kept with the HIRES PRV Python access toolkit in GitHub: https://github.com/Caltech-IPAC/hiresprv

You can download and interact with your own copy of this notebook here

Ancillary Tools¶

Workspace Directory Listing¶

We can get a list of all the downloadable files in the workspace. There are utility functions (below) for quickly downloading the most-useful final products but we might want to see the intermediate products to evaluate/diagnose the processing. See http://caltech-ipac.github.io/hiresprv/outputs.html#workspace-directory-structure for a description of the files availble in each directory.

[2]:

import json
import os
from hiresprv.download import Download

data = Download('prv.cookies', './', debug=True)

listing = data.directory_listing()
prtstring = json.dumps(listing, indent=4, sort_keys=True)[:2000]
prtstring += "\n... output truncated ..."

print(prtstring)

{
    "database": "prvState.db",
    "deblazed": [
        "deblazed/r20091231.160.fits",
        "deblazed/r20130629.110.fits",
        "deblazed/r20091231.287.fits",
        "deblazed/r20130912.73.fits",
        "deblazed/r20130912.1.fits",
        "deblazed/r20130912.128.fits",
        "deblazed/r20150606.157.fits",
        "deblazed/r20091231.225.fits",
        "deblazed/r20130912.163.fits",
        "deblazed/r20091231.189.fits",
        "deblazed/r20130629.92.fits",
        "deblazed/r20150606.97.fits",
        "deblazed/r20130912.152.fits",
        "deblazed/r20091231.214.fits",
        "deblazed/r20150606.166.fits",
        "deblazed/r20091231.67.fits",
        "deblazed/r20130912.119.fits",
        "deblazed/r20091231.151.fits",
        "deblazed/r20130912.130.fits",
        "deblazed/r20150606.104.fits",
        "deblazed/r20091231.276.fits",
        "deblazed/r20091231.191.fits",
        "deblazed/r20130912.82.fits",
        "deblazed/r20130629.19.fits",
        "deblazed/r20091231.178.fits",
        "deblazed/r20091231.133.fits",
        "deblazed/r20130629.108.fits",
        "deblazed/r20130912.11.fits",
        "deblazed/r20130629.3.fits",
        "deblazed/r20091231.96.fits",
        "deblazed/r20091231.102.fits",
        "deblazed/r20150606.66.fits",
        "deblazed/r20091231.3.fits",
        "deblazed/r20091231.149.fits",
        "deblazed/r20091231.247.fits",
        "deblazed/r20150606.135.fits",
        "deblazed/r20130912.101.fits",
        "deblazed/r20130629.96.fits",
        "deblazed/r20150606.129.fits",
        "deblazed/r20130912.156.fits",
        "deblazed/r20150606.162.fits",
        "deblazed/r20091231.210.fits",
        "deblazed/r20150606.93.fits",
        "deblazed/r20091231.155.fits",
        "deblazed/r20130629.114.fits",
        "deblazed/r20091231.283.fits",
        "deblazed/r20091231.164.fits",
        "deblazed/r20130912.5.fits",
        "deblazed/r20130912.77.fits",
        "deblazed/r20091231.221.fits",
        "deblazed/r
... output truncated ...

File Download¶

Any of the files in the listing can be retrieved by name. We specify the output directory when initializing the Download object.

[3]:

rtn = data.download("deblazed/r20091231.247.fits")

print(rtn)

{'status': 'ok', 'msg': ''}

Augmented Processing¶

There are no tunable parameters associated with the HIRES PRV pipeline. However, you can affect the processing in a couple of places by “removing” files (usually because there is something suspect about them). For instance, a bad B-star observation can adversely affect the stellar template calculation so you may want to rerun the template building without it. We don’t actually delete the file; we deactivate it but leave it in place in case you change your mind.

Similarly, in making the final RV curve you may decide to remove one or more of the reduced RV measurements.

Deactivating Files¶

Sometimes it turns out that a data file is suboptimal and should probably be removed from the processing. It might be a B-star observation bracketing a template measurement or one of the RV observations that should be removed from an RV curve.

The PRV service takes responsibility for remembering what files have been processed and for providing a mechanism (the “DEACTIVATED” column in the database) for “turning off” specific files. It does not automatically redo all the affected downstream processing. That is left up to the user so it is best to take care of all of that as soon as possible.

In this example, we will turn off one of the B-star observations. This requires regenerating the template(s) associated with that observation:

[14]:

from hiresprv.idldriver import Idldriver

idl = Idldriver('prv.cookies')

rtn = idl.run_script("""
deactivate r20091231.77
template 185144 20091231
""")

print(rtn)

status= ok
msg= Script running in background. Consult monitor for status.
None

Having regenerated the template, we must therefore regenerate the downstream data (RVs and RV curve). The underlying processing code will not regenerate the reduced RV data if that file already exists so we need to explicitly deactivate the reduced RV files if we intend to re-calculate them:

[4]:

from hiresprv.idldriver import Idldriver

idl = Idldriver('prv.cookies')

rtn = idl.run_script("""
deactivate vdaa185144_r20091231.72
deactivate vdaa185144_r20091231.73
deactivate vdaa185144_r20091231.74
deactivate vdaa185144_r20150606.145
deactivate vdaa185144_r20150606.146
deactivate vdaa185144_r20150606.147
rv 185144 r20091231.72
rv 185144 r20091231.73
rv 185144 r20091231.74
rv 185144 r20150606.145
rv 185144 r20150606.146
rv 185144 r20150606.147
rvcurve 185144
""")

print(rtn)

status= ok
msg= Script running in background. Consult monitor for status.
None

After further consideration, we decide that the second RV measurement was of inferior quality, so we deactivate it, too. You can find the name of the reduced RV file by listing out the full database again or by doing a database search (if you look a the file name, it follows a simple pattern):

[4]:

from hiresprv.database import Database
import pandas as pd

state = Database('prv.cookies')

url = state.search(sql="select DATE, OBTYPE, FILENAME, TARGET, BJD, BCVEL from FILES where TARGET like 'rv data';")

df = pd.read_html(url, header=0)[0]
df.head(15)

[4]:

	OBTYPE	FILENAME	TARGET	BJD
0	Reduced RV	vdaa185144_r20091231.72	rv data	2440000.0
1	Reduced RV	vdaa185144_r20091231.73	rv data	2440000.0
2	Reduced RV	vdaa185144_r20091231.74	rv data	2440000.0
3	Reduced RV	vdaa185144_r20150606.145	rv data	2440000.0
4	Reduced RV	vdaa185144_r20150606.146	rv data	2440000.0
5	Reduced RV	vdaa185144_r20150606.147	rv data	2440000.0

[6]:

from hiresprv.idldriver import Idldriver

idl = Idldriver('prv.cookies')

rtn = idl.run_script("""
deactivate vdaa185144_r20091231.73
rvcurve 185144
""")

print(rtn)

status= ok
msg= Script running in background. Consult monitor for status.
None

So now we have only five reduced RV measurements and they were reduced with a slightly different template measurement. This is reflected in the RV curve, which you can compare to the original:

[8]:

from hiresprv.download import Download

data = Download('prv.cookies', './')

rtn = data.rvcurve('185144')

with open('vst185144.csv', 'r') as file:
  for line in file:
    print(line, end='')

BJD_TDB,RV,RV_ERR,BC,ADU,CHI2
15196.69208800001,-2.176984931266687,0.782362,-4620.095214843750,52362,1.04989
15196.69329199987,2.082068283340939,0.778802,-4620.320800781250,48950,1.05804
17180.10972899990,-2.351030457353758,0.769511,3189.794921875000,55029,1.10542
17180.11030799989,1.241724284090688,0.678055,3189.327880859375,55593,1.10771
17180.11088699987,0.9359797393606732,0.686433,3188.863037109375,48769,1.10120