Welcome to voeventdb.remote’s documentation!

Version 0

Contents:

Introduction

voeventdb.remote is a Python library for performing remote queries against a voeventdb REST interface, like the one hosted by the 4 Pi Sky research group at http://voeventdb.4pisky.org.

You can use this library to easily retrieve information about transient astronomical events, if that information was sent via the public VOEvent network.

For a fuller introduction, see the main voeventdb introduction. Alternatively, jump right in to the tutorial to learn by example.

Installation

voeventdb.remote is available via PyPi, which means that if you are using a virtualenv then you should be able to simply type:

pip install voeventdb.remote

at the command line. Alternatively, you can install into your user-area using:

pip install --user voeventdb.remote

Tutorial

The tutorials take the form of Jupyter notebooks; you can either read them online or download them to work with interactively. The notebook files can be found in the notebooks folder of the source repository, or downloaded using the links on each tutorial page.

Tutorial 1

Right click here and 'save as' to download this notebook.

Getting started with voeventdb.remote

In this notebook, we will cover the basics of working with voeventdb.remote, and give a quick tour of the sorts of data you can retrieve.

First, let's set up logging - by switching on 'DEBUG' level logging, we get to see the HTTP requests that are being made behind the scenes every time you run a query using voeventdb.remote. (NB we'll be using the Python-3 style print operator.)

In [1]:
from __future__ import print_function
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

Next, library imports: Most of the user-facing code lives in voeventdb.remote.apiv1, but we'll also alias voeventdb.remote to the shorthand vr:

In [2]:
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1

Aside: Autocompletion tricks

For those new to IPython / notebooks: note you can use tab-completion to help you explore / save on typing (as long as you've already imported the relevant packages). For example, type:

apiv1.

in a notebook code-cell or IPython terminal and then hit Tab to see your available options. Likewise, in notebooks you can position your cursor on a function and hit Shift+Tab to see a summary of function arguments, etc.


There are a couple of variables living in the top-level voeventdb.remote module; for example the default_host. You can set this to avoid specifying a host for every query. By default it's set to point at the 4PiSky server:

In [3]:
# For testing against a local-dev server:
# vr.default_host = 'http://localhost:5000'
print(vr.default_host)
http://voeventdb.4pisky.org

OK, let's query the database. We use different HTTP addresses, called endpoints, to give us different summaries or subsets of the database. We'll start off by using apiv1.count to find out how many packets are stored in this database - if it's empty, any other queries won't do us much good:

In [4]:
apiv1.count()
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count HTTP/1.1" 200 125
Out[4]:
865272

Getting on for a million packets, that's quite lot! So there's plenty here for us to work with.

IVORN's and single-packet details

Next let's try apiv1.list_ivorn, which returns a list of IVORN's, unique idenitifiers which represent a VOEvent packet, analogous to how a URL represents a web-page.

In [5]:
ivorn_list = apiv1.list_ivorn()
len(ivorn_list)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count HTTP/1.1" 200 125
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=1000&offset=0 HTTP/1.1" 200 68334
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=1000&offset=1000 HTTP/1.1" 200 67397
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=1000&offset=2000 HTTP/1.1" 200 66248
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=1000&offset=3000 HTTP/1.1" 200 66248
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=1000&offset=4000 HTTP/1.1" 200 66248
Out[5]:
5000
In [6]:
#Take a look at the first 10
ivorn_list[:10]
Out[6]:
[u'ivo://nasa.gsfc.gcn/Fermi#GBM_Alert_2015-11-19T17:05:04.87_469645508_1-285',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_46-286',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_57-287',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_68-289',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_72-293',
 u'ivo://nasa.gsfc.gcn/SWIFT#Point_Dir_2015-11-19T17:07:00.00_436241615-294',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_16811395-301',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_16811397-314',
 u'ivo://nasa.gsfc.gcn/Fermi#Point_Dir_2015-11-19T17:13:00.00_000000-0-315',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_16811398-328']

We got back a list of IVORN's as expected - but only 5000 of them, not a million. By default, voeventdb.remote applies a limit to the maximum number of entries returned by a list query, as defined in default_list_n_max:

In [7]:
print(vr.default_list_n_max)
5000

You can override this, or set it to 0 to simply fetch all matching results (though this can take a while).

In [8]:
## How to override the default:
# vr.default_list_n_max = 2500
## Set n_max for a single query:
short_list = apiv1.list_ivorn(n_max=50)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count HTTP/1.1" 200 125
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=50&offset=0 HTTP/1.1" 200 3772

If we know the IVORN of a packet, we can look up some details using the apiv1.packet_synopsis endpoint:

In [9]:
ivorn = ivorn_list[0]
apiv1.packet_synopsis(ivorn)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/packet/synopsis/ivo%3A%2F%2Fnasa.gsfc.gcn%2FFermi%23GBM_Alert_2015-11-19T17%3A05%3A04.87_469645508_1-285 HTTP/1.1" 200 679
Out[9]:
{u'coords': [],
 u'refs': [],
 u'relevant_urls': [],
 u'voevent': {u'author_datetime': u'2015-11-19T17:05:09+00:00',
  u'author_ivorn': u'ivo://nasa.gsfc.tan/gcn',
  u'ivorn': u'ivo://nasa.gsfc.gcn/Fermi#GBM_Alert_2015-11-19T17:05:04.87_469645508_1-285',
  u'received': u'2015-11-19T17:05:11.609442+00:00',
  u'role': u'observation',
  u'stream': u'nasa.gsfc.gcn/Fermi',
  u'version': u'2.0'}}

And we can even retrieve the raw XML, (i.e. the original packet), so we can pull out any data specific to a given observatory, etc:

In [10]:
xml = apiv1.packet_xml(ivorn)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/packet/xml/ivo%3A%2F%2Fnasa.gsfc.gcn%2FFermi%23GBM_Alert_2015-11-19T17%3A05%3A04.87_469645508_1-285 HTTP/1.1" 200 1451

A convenient way to inspect the xml-data is to use the voevent-parse library (which even has its very own tutorial).

In [11]:
## A brief example, see the voevent-parse tutorial for more
# import voeventparse as vp
# voe_tree = vp.loads(xml)
# vp.pull_params(voe_tree)

Summary endpoints

So we can view individual packet IVORNS, and look up packet details, but that's very fine-grained information - we don't want to wade through all of them to figure out what's in the database. We can use 'summary' endpoints to give us a higher-level overview. For example, we can organise VOEvents by 'streams' - basically the bit of the IVORN between the ivo:// prefix and the # symbol. The apiv1.map_stream_count endpoint tells us how many VOEvent packets belong to each stream:

In [12]:
apiv1.map_stream_count()
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count HTTP/1.1" 200 1023
Out[12]:
{u'com.dc3/dc3.broker': 22570,
 u'nasa.gsfc.gcn/AGILE': 4100,
 u'nasa.gsfc.gcn/COUNTERPART': 72,
 u'nasa.gsfc.gcn/Fermi': 28154,
 u'nasa.gsfc.gcn/GRO': 4361,
 u'nasa.gsfc.gcn/HETE': 4027,
 u'nasa.gsfc.gcn/INTEGRAL': 23158,
 u'nasa.gsfc.gcn/IPN': 345,
 u'nasa.gsfc.gcn/KONUS': 308,
 u'nasa.gsfc.gcn/MAXI': 4246,
 u'nasa.gsfc.gcn/MOA': 969,
 u'nasa.gsfc.gcn/SNEWS': 2,
 u'nasa.gsfc.gcn/SUZAKU': 17,
 u'nasa.gsfc.gcn/SWIFT': 769627,
 u'nvo.caltech/voeventnet/catot': 66,
 u'nvo.caltech/voeventnet/mlsot': 147,
 u'svomcgft.naoc/VOEVENTTEST': 1768,
 u'voevent.4pisky.org/TEST': 10,
 u'voevent.4pisky.org/voevent-broadcast': 598,
 u'voevent.4pisky.org/voevent-receive': 598,
 u'voevent.astro.soton/TEST': 13,
 u'voevent.astro.soton/TESTRESPONSE': 13,
 u'voevent.organization.tld/TEST': 22,
 u'voevent.phys.soton.ac.uk/AMI-REQUEST': 83}

Alternatively, we can slice up the packets by the authoring timestamp, binning by calendar month:

In [13]:
apiv1.map_authored_month_count()
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/authored_month_count HTTP/1.1" 200 657
Out[13]:
{u'2005-04': 1,
 u'2014-04': 14196,
 u'2014-05': 50905,
 u'2014-06': 46256,
 u'2014-07': 41311,
 u'2014-08': 40602,
 u'2014-09': 47832,
 u'2014-10': 47692,
 u'2014-11': 37713,
 u'2014-12': 35695,
 u'2015-01': 47874,
 u'2015-02': 39768,
 u'2015-03': 50419,
 u'2015-04': 44248,
 u'2015-05': 41060,
 u'2015-06': 42728,
 u'2015-07': 51377,
 u'2015-08': 49536,
 u'2015-09': 49201,
 u'2015-10': 41849,
 u'2015-11': 29448,
 u'2015-12': 15540,
 u'null': 24}

Or, we can divide them up by assigned role, which comes in three flavours:

In [14]:
apiv1.map_role_count()
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/role_count HTTP/1.1" 200 209
Out[14]:
{u'observation': 729662, u'test': 60262, u'utility': 75353}

Similarly, apiv1.map_stream_role_count gives a breakdown of how many packets belong to each role, listed by stream.

Coming next ...

So now that we have an idea of what's in the database, how do we choose a category and drill-down to select specific VOEvents? For that we need filters, which are covered in tutorial 2.

Tutorial 2

Right click here and 'save as' to download this notebook.

Narrowing your query and digging into your results

We previously covered the basics of working with voeventdb.remote in tutorial 1.

In this notebook, we'll demonstrate how to use combinations of filters to narrow down your query, and demonstrate the 'helper classes' you can use to easily access the details of a particular packet.

As before, we'll switch on 'DEBUG' level logging, to see the the HTTP requests go whizzing by.

In [1]:
from __future__ import print_function
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
In [2]:
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1

We've already briefly looked at the map_stream_count endpoint, and mentioned how VOEvents come in three flavours of role, 'observation', 'utility', and 'test'. Let's remind ourselves what the default map_stream_count output looks like:

In [3]:
apiv1.map_stream_count()
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count HTTP/1.1" 200 1023
Out[3]:
{u'com.dc3/dc3.broker': 22570,
 u'nasa.gsfc.gcn/AGILE': 4100,
 u'nasa.gsfc.gcn/COUNTERPART': 72,
 u'nasa.gsfc.gcn/Fermi': 28154,
 u'nasa.gsfc.gcn/GRO': 4361,
 u'nasa.gsfc.gcn/HETE': 4027,
 u'nasa.gsfc.gcn/INTEGRAL': 23158,
 u'nasa.gsfc.gcn/IPN': 345,
 u'nasa.gsfc.gcn/KONUS': 308,
 u'nasa.gsfc.gcn/MAXI': 4246,
 u'nasa.gsfc.gcn/MOA': 969,
 u'nasa.gsfc.gcn/SNEWS': 2,
 u'nasa.gsfc.gcn/SUZAKU': 17,
 u'nasa.gsfc.gcn/SWIFT': 769618,
 u'nvo.caltech/voeventnet/catot': 66,
 u'nvo.caltech/voeventnet/mlsot': 147,
 u'svomcgft.naoc/VOEVENTTEST': 1768,
 u'voevent.4pisky.org/TEST': 10,
 u'voevent.4pisky.org/voevent-broadcast': 598,
 u'voevent.4pisky.org/voevent-receive': 598,
 u'voevent.astro.soton/TEST': 13,
 u'voevent.astro.soton/TESTRESPONSE': 13,
 u'voevent.organization.tld/TEST': 22,
 u'voevent.phys.soton.ac.uk/AMI-REQUEST': 83}

Using filters

Quite obviously, a number of those streams are 'junk', they contain only test-packets used to verify that the VOEvent infrastructure is up and working correctly. For scientific work, we'll want to filter those out.

Fortunately, we can ask the voeventdb server to do the filtering work for us. The voeventdb.remote library comes with an easy-to-use list of filters, stored as voeventdb.remote.apiv1.FilterKeys. To see what's available at a glance you can use the IPython tab-completion and doc-lookup tools, as in the cell below.

Full definitions of the filter-keys (and example filter-values) can be found in the voeventdb server docs, but we'll cover most of them in these tutorial notebooks - read on.

In [4]:
#Alias voeventdb.remote.apiv1.FilterKeys to just 'FilterKeys', for brevity
from voeventdb.remote.apiv1 import FilterKeys
In [5]:
## To see the list of filters, you can use tab-completion:
## (Uncomment the following line and try it for yourself)
# FilterKeys.
## Or the ipython doc-lookup magic, by prefixing with ``??`` and running the cell:
# ??FilterKeys

Filtering by role

So: we were trying to filter out the test-packets. FilterKeys.role sounds promising. To apply a filter, or multiple filters, we simply define a dictionary with the filters we want to apply, and then pass it to the relevant query-function, like this:

In [6]:
my_filters = { FilterKeys.role: 'observation' }
In [7]:
apiv1.map_stream_count(my_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?role=observation HTTP/1.1" 200 581
Out[7]:
{u'nasa.gsfc.gcn/COUNTERPART': 72,
 u'nasa.gsfc.gcn/Fermi': 5904,
 u'nasa.gsfc.gcn/INTEGRAL': 802,
 u'nasa.gsfc.gcn/IPN': 345,
 u'nasa.gsfc.gcn/KONUS': 308,
 u'nasa.gsfc.gcn/MAXI': 193,
 u'nasa.gsfc.gcn/MOA': 969,
 u'nasa.gsfc.gcn/SUZAKU': 17,
 u'nasa.gsfc.gcn/SWIFT': 720827,
 u'nvo.caltech/voeventnet/catot': 66,
 u'nvo.caltech/voeventnet/mlsot': 147}

Filtering by date

That results in a much shorter list, containing only scientifically interesting streams. Still, those numbers are pretty large (mainly for Swift). It might be useful to get a smaller representative sample. How many packets will we get if we limit our query to a single week?

In [8]:
from datetime import datetime, timedelta
import pytz
start_date = datetime(2015,12,1,tzinfo=pytz.UTC)
my_filters = { 
    FilterKeys.role: 'observation',
    FilterKeys.authored_since: start_date,
    FilterKeys.authored_until: start_date + timedelta(days=7)
    }
my_filters
Out[8]:
{'authored_since': datetime.datetime(2015, 12, 1, 0, 0, tzinfo=<UTC>),
 'authored_until': datetime.datetime(2015, 12, 8, 0, 0, tzinfo=<UTC>),
 'role': 'observation'}
In [9]:
apiv1.map_stream_count(my_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&role=observation&authored_until=2015-12-08T00%3A00%3A00%2B00%3A00 HTTP/1.1" 200 617
Out[9]:
{u'nasa.gsfc.gcn/Fermi': 44,
 u'nasa.gsfc.gcn/INTEGRAL': 1,
 u'nasa.gsfc.gcn/IPN': 4,
 u'nasa.gsfc.gcn/KONUS': 4,
 u'nasa.gsfc.gcn/MAXI': 3,
 u'nasa.gsfc.gcn/SWIFT': 2629}

Filtering by stream

Ok, so there's still a lot of Swift packets there. Let's take a look at a sample of those, and see if we can break them up further. First, lets add another filter to limit our query to just Swift packets.

In [10]:
my_filters[FilterKeys.stream] = 'nasa.gsfc.gcn/SWIFT'
my_filters
Out[10]:
{'authored_since': datetime.datetime(2015, 12, 1, 0, 0, tzinfo=<UTC>),
 'authored_until': datetime.datetime(2015, 12, 8, 0, 0, tzinfo=<UTC>),
 'role': 'observation',
 'stream': 'nasa.gsfc.gcn/SWIFT'}

So now if we apply the filters to map_stream_count, we only get back one entry (the Swift stream):

In [11]:
apiv1.map_stream_count(filters=my_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&role=observation&authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT HTTP/1.1" 200 546
Out[11]:
{u'nasa.gsfc.gcn/SWIFT': 2629}

Filters can be used across different query-endpoints

Not particularly helpful, but at least everything is working as expected. Now, the neat thing about the voeventdb filters is that they can be applied to any query-endpoint - we can just re-use the filter-dictionary with the apiv1.list_ivorn function to get back a list of IVORNs:

In [12]:
swift_ivorns = apiv1.list_ivorn(filters=my_filters)
print("Retrieved",len(swift_ivorns),"IVORNs")
#Show just the first 10
swift_ivorns[:10]
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&role=observation&authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT HTTP/1.1" 200 491
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT&authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&limit=1000&offset=0&role=observation HTTP/1.1" 200 67463
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT&authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&limit=1000&offset=1000&role=observation HTTP/1.1" 200 66412
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT&authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&limit=1000&offset=2000&role=observation HTTP/1.1" 200 41330
Retrieved 2629 IVORNs
Out[12]:
[u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743023201-093',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743023206-097',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743023841-098',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743023841-099',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743024161-100',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Known_Pos_1743024201-101',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743024201-102',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743024481-103',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1743025086-104',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Known_Pos_1743025796-105']

That's a long list, but there's clearly a pattern to how the Swift IVORNs are formatted. We'll use a little Python trickery (cf set, str.rsplit) to chop off the trailing ID numbers and sort them into sub-categories:

In [13]:
swift_categories = set(ivorn.rsplit('_',1)[0] for ivorn in swift_ivorns)
swift_categories
Out[13]:
{u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Known_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_QuickLook_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Scaledmap',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Nack_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec'}

Now we're getting somewhere! We can clearly see the subcategories of Swift packets - BAT alerts, XRT positions, UVOT followup, etc.

Filtering by IVORN substring

We can use this knowledge to refine our filters, by filtering on a substring of the IVORN, using the ivorn_contains filter. For example, we might want to filter to just those IVORNs containing XRT positions (note this filter is case-sensitive):

In [14]:
my_filters[FilterKeys.ivorn_contains] = 'XRT_Pos'
my_filters
Out[14]:
{'authored_since': datetime.datetime(2015, 12, 1, 0, 0, tzinfo=<UTC>),
 'authored_until': datetime.datetime(2015, 12, 8, 0, 0, tzinfo=<UTC>),
 'ivorn_contains': 'XRT_Pos',
 'role': 'observation',
 'stream': 'nasa.gsfc.gcn/SWIFT'}
In [15]:
xrt_pos_ivorns = apiv1.list_ivorn(filters=my_filters)
print("Retrieved",len(xrt_pos_ivorns),"IVORNs")
xrt_pos_ivorns
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&role=observation&authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT&ivorn_contains=XRT_Pos HTTP/1.1" 200 558
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?authored_until=2015-12-08T00%3A00%3A00%2B00%3A00&stream=nasa.gsfc.gcn%2FSWIFT&authored_since=2015-12-01T00%3A00%3A00%2B00%3A00&limit=2&offset=0&role=observation&ivorn_contains=XRT_Pos HTTP/1.1" 200 773
Retrieved 2 IVORNs
Out[15]:
[u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605']

As in tutorial 1, we can inspect the details of any given packet using the packet_synopsis endpoint - we'll take a look at the first one. This packet makes a good example, as it includes details of the event co-ordinates and timestamp, and also references an earlier VOEvent:

In [16]:
synopsis_dict = apiv1.packet_synopsis(xrt_pos_ivorns[0])
synopsis_dict
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/packet/synopsis/ivo%3A%2F%2Fnasa.gsfc.gcn%2FSWIFT%23XRT_Pos_666352-553 HTTP/1.1" 200 1092
Out[16]:
{u'coords': [{u'dec': 35.7445,
   u'error': 0.0014,
   u'ra': 229.2874,
   u'time': u'2015-12-05T15:47:32.740000+00:00'}],
 u'refs': [{u'cite_type': u'followup',
   u'description': u'This is the XRT Position for the original BAT trigger.',
   u'ref_ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547'}],
 u'relevant_urls': [u'http://www.swift.ac.uk/search/summary.php?obsid=666352',
  u'http://gcn.gsfc.nasa.gov/other/666352.swift'],
 u'voevent': {u'author_datetime': u'2015-12-05T15:48:53+00:00',
  u'author_ivorn': u'ivo://nasa.gsfc.tan/gcn',
  u'ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
  u'received': u'2015-12-05T15:48:57.012291+00:00',
  u'role': u'observation',
  u'stream': u'nasa.gsfc.gcn/SWIFT',
  u'version': u'2.0'}}

Ready-made 'helper' classes for parsing output

Nested dictionaries can be kind of a pain to work with. If you want, you can use voeventdb.remote's Synopsis 'helper' class to parse this into an easy-to use object.

In [17]:
from voeventdb.remote.helpers import Synopsis
In [18]:
xrt_synopsis = Synopsis(synopsis_dict)
# Prints with nicer formatting, ordering of values:
print(xrt_synopsis)
{ 'author_datetime': '2015-12-05T15:48:53+00:00',
  'author_ivorn': u'ivo://nasa.gsfc.tan/gcn',
  'ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
  'received': '2015-12-05T15:48:57.012291+00:00',
  'references': [ { u'cite_type': u'followup',
                    u'description': u'This is the XRT Position for the original BAT trigger.',
                    u'ref_ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547'}],
  'role': u'observation',
  'sky_events': [ <SkyCoord (ICRS): (ra, dec) in deg
    (229.2874, 35.7445)> +/- 0.0014 @ 2015-12-05T15:47:32.740000+00:00],
  'stream': u'nasa.gsfc.gcn/SWIFT',
  'version': u'2.0'}

Now we can easily access the values (with the ever-handy IPython autocompletion):

In [19]:
xrt_synopsis.author_ivorn
Out[19]:
u'ivo://nasa.gsfc.tan/gcn'
In [20]:
xrt_synopsis.references
Out[20]:
[{u'cite_type': u'followup',
  u'description': u'This is the XRT Position for the original BAT trigger.',
  u'ref_ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547'}]

One of the Synopsis class attributes is a list called sky_events. Each entry is a SkyEvent class, which reprents a very basic set of information about an observed event:

  • estimated position,
  • error circle on the estimated position,
  • timestamp of the observed event.

The position coordinates and error-circle are represented by astropy.coordinates classes, which come with a bunch of features related to formatting, distance calculations, frame-of-reference transformations, etc.

In [21]:
xrt_synopsis.sky_events
Out[21]:
[<SkyCoord (ICRS): (ra, dec) in deg
     (229.2874, 35.7445)> +/- 0.0014 @ 2015-12-05T15:47:32.740000+00:00]
In [22]:
# List of 1, in this case. Grab the first (and only) element:
sky_event = xrt_synopsis.sky_events[0]
In [23]:
print(type(sky_event.position))
sky_event.position
<class 'astropy.coordinates.sky_coordinate.SkyCoord'>
Out[23]:
<SkyCoord (ICRS): (ra, dec) in deg
    (229.2874, 35.7445)>
In [24]:
print(type(sky_event.position_error))
sky_event.position_error.deg
<class 'astropy.coordinates.angles.Angle'>
Out[24]:
0.0014

Astropy coordinates come with all the usual weird and wonderful astronomical formatting options, see the astropy docs for details:

In [25]:
print(sky_event.position.ra.deg)
print(sky_event.position.ra.hms)
229.2874
hms_tuple(h=15.0, m=17.0, s=8.9760000000072182)

Advanced usage: specifying multiple values for the same filter

Before we move on, it's worth mentioning that some filters can take on multiple values. This is specified by defining the filter-value as a list - for example, to return all VOEvents with a role of 'observation' or 'utility' we can use the following:

In [26]:
my_filters = {apiv1.FilterKeys.role: ['observation','utility']}
apiv1.map_stream_count(my_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?role=observation&role=utility HTTP/1.1" 200 662
Out[26]:
{u'nasa.gsfc.gcn/COUNTERPART': 72,
 u'nasa.gsfc.gcn/Fermi': 19995,
 u'nasa.gsfc.gcn/INTEGRAL': 17303,
 u'nasa.gsfc.gcn/IPN': 345,
 u'nasa.gsfc.gcn/KONUS': 308,
 u'nasa.gsfc.gcn/MAXI': 193,
 u'nasa.gsfc.gcn/MOA': 969,
 u'nasa.gsfc.gcn/SUZAKU': 17,
 u'nasa.gsfc.gcn/SWIFT': 765510,
 u'nvo.caltech/voeventnet/catot': 66,
 u'nvo.caltech/voeventnet/mlsot': 147,
 u'voevent.phys.soton.ac.uk/AMI-REQUEST': 83}

How does this work? Well, we can think of each entry in the list defining a separate filter. For the role value, these filters are combined in the logical 'OR' sense, so we get back combined counts for both 'observation' and 'utility' packets. You can check whether a filter accepts multiple values, and if they are combined via logical 'OR' or 'AND', by checking the filter-definitions page and looking for the combinator attribute.

(If this sounds confusing, don't worry - it can safely be ignored unless you're planning to use really tricky queries.)

Coming next ...

We've seen how to narrow our search, locate packets of interest, and use helper-classes to easily access packet details. In tutorials 3 & 4, we'll cover different ways of finding related VOEvents.

Tutorial 3

Right click here and 'save as' to download this notebook.

Spatial queries

By now, you should have some idea of how to explore the various voeventdb.remote endpoints and apply filters.

In this notebook, we'll extend our filter-toolset to include cone-searches.

In [1]:
from __future__ import print_function
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
In [2]:
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1
from voeventdb.remote.apiv1 import FilterKeys
from voeventdb.remote.helpers import Synopsis
from astropy.coordinates import Angle, SkyCoord

In the last tutorial, we inspected a Swift XRT event which gave us a position estimate and and error-circle. Let's grab those details again:

In [3]:
xrt_synopsis = Synopsis(apiv1.packet_synopsis('ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553'))
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/packet/synopsis/ivo%3A%2F%2Fnasa.gsfc.gcn%2FSWIFT%23XRT_Pos_666352-553 HTTP/1.1" 200 1092
In [4]:
sky_event = xrt_synopsis.sky_events[0]
print(sky_event)
print("Error-circle radius in degrees, arcminutes and arcseconds:", sky_event.position_error)
<SkyCoord (ICRS): (ra, dec) in deg
    (229.2874, 35.7445)> +/- 0.0014 @ 2015-12-05T15:47:32.740000+00:00
Error-circle radius in degrees, arcminutes and arcseconds: 0d00m05.04s

Filtering using a cone-search (AKA spatial queries)

Next, let's see if there are any other recorded events with associated positions nearby. To do so, we'll need to define a 'cone', a sky-position and circle around it to search in. For setting up a voeventdb.remote cone-filter, we can use a tuple of type

(astropy.coordinates.SkyCoord, astropy.coordinates.Angle)

A natural candidate is the position and position-error from the XRT event; like so:

In [5]:
cone = (sky_event.position, sky_event.position_error)
cone
Out[5]:
(<SkyCoord (ICRS): (ra, dec) in deg
     (229.2874, 35.7445)>, <Angle 0.0014 deg>)

However, the XRT position has a really tight error-circle, about 5 arcseconds. Note that the cone-search will only return VOEvents with a best-estimate position within the cone - it does not take into account overlapping error-circles (at least for version 1!). This means that we could have a related event with a large error-circle, but which has a best-estimate position just outside the tiny XRT error-circle, and it wouldn't be returned - so we have to use some judgement here. We'll set the cone angular radius to half a degree, instead:

In [6]:
cone = (sky_event.position, Angle(0.5, unit='deg'))
cone
Out[6]:
(<SkyCoord (ICRS): (ra, dec) in deg
     (229.2874, 35.7445)>, <Angle 0.5 deg>)

OK, let's see how that works:

In [7]:
cone_filters = {
    FilterKeys.role: 'observation',
    FilterKeys.cone: cone
    }
In [8]:
apiv1.map_stream_count(cone_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?role=observation&cone=%5B229.2874%2C+35.7445%2C+0.5%5D HTTP/1.1" 200 324
Out[8]:
{u'nasa.gsfc.gcn/SWIFT': 37}

A reasonable number. Let's take a look:

In [9]:
sorted(apiv1.list_ivorn(cone_filters))
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?role=observation&cone=%5B229.2874%2C+35.7445%2C+0.5%5D HTTP/1.1" 200 269
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=37&role=observation&cone=%5B229.2874%2C+35.7445%2C+0.5%5D&offset=0 HTTP/1.1" 200 2591
Out[9]:
[u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Known_Pos_1745116191-419',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-562',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-563',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-599',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-625',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-633',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_QuickLook_Pos_666352-546',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1519728446-452',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1529290721-073',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1599096481-477',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1599096481-752',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1705123121-317',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745115961-415',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745116036-416',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745116081-418',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_666352-551',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_17443568-112',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_666352-552',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-574',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-611',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos_666352-636',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-578',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-612',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-570',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-604',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-569',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-600',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image_666352-554',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve_666352-561',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image_666352-555',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-560',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-568',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-559',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-564']

Investigating 'background' event rates

So, we have a bunch of Swift packets related to the same event (ID 666352), some 'BAT_SubSubThresh' events, and a 'Known_Pos' event. It's worth noting that 'SubSubThresh' events are extremely common and show up all over; in fact they currently make up about half the packets in the database:

In [10]:
all_events_count = apiv1.count(filters={FilterKeys.role:'observation'})
ss_thresh_count = apiv1.count(filters={
        FilterKeys.role:'observation', 
        FilterKeys.ivorn_contains:'BAT_SubSubThresh',
        })
print("Of {} observation packets in the database, {} are BAT_SubSubThresh packets".format(
        all_events_count, ss_thresh_count))
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?role=observation HTTP/1.1" 200 185
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?role=observation&ivorn_contains=BAT_SubSubThresh HTTP/1.1" 200 273
Of 729673 observation packets in the database, 351716 are BAT_SubSubThresh packets

So it's perhaps not surprising that we'd encounter a few of them co-incidentally lying in the search-cone. We can define a search-cone of the same radius at arbitrary co-ordinates and see what we get back, for comparison:

In [11]:
cone2 = (SkyCoord(ra=0, dec=35., unit='deg'), Angle(0.5, unit='deg'))
cone2
Out[11]:
(<SkyCoord (ICRS): (ra, dec) in deg
     (0.0, 35.0)>, <Angle 0.5 deg>)
In [12]:
cone2_filters = {
    FilterKeys.role: 'observation',
    FilterKeys.cone: cone2
    }
In [13]:
apiv1.map_stream_count(filters=cone2_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/map/stream_count?role=observation&cone=%5B0.0%2C+35.0%2C+0.5%5D HTTP/1.1" 200 307
Out[13]:
{u'nasa.gsfc.gcn/SWIFT': 9}
In [14]:
apiv1.list_ivorn(filters=cone2_filters)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?role=observation&cone=%5B0.0%2C+35.0%2C+0.5%5D HTTP/1.1" 200 252
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?limit=9&role=observation&cone=%5B0.0%2C+35.0%2C+0.5%5D&offset=0 HTTP/1.1" 200 984
Out[14]:
[u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1675655366-828',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1610834361-085',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1627263451-292',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1604144441-970',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1618893046-031',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1506180871-332',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1595742726-128',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1705707921-650',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1706782731-079']

Result: looks like we get a similar number of these 'SubSubThresh' events wherever we point our search-cone!

Restricting the time-period to a 60-day window

Fortunately, it's easy to narrow things down a bit further, if we expect related events to occur within a reasonably short time-interval. Let's modify our original filter-set a bit more:

In [15]:
from datetime import timedelta
cone_filters[FilterKeys.authored_since] = sky_event.timestamp - timedelta(days=30)
cone_filters[FilterKeys.authored_until] = sky_event.timestamp + timedelta(days=30)
cone_filters
Out[15]:
{'authored_since': datetime.datetime(2015, 11, 5, 15, 47, 32, 740000, tzinfo=<FixedOffset u'+00:00' datetime.timedelta(0)>),
 'authored_until': datetime.datetime(2016, 1, 4, 15, 47, 32, 740000, tzinfo=<FixedOffset u'+00:00' datetime.timedelta(0)>),
 'cone': (<SkyCoord (ICRS): (ra, dec) in deg
      (229.2874, 35.7445)>, <Angle 0.5 deg>),
 'role': 'observation'}
In [16]:
sorted(apiv1.list_ivorn(cone_filters))
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/count?authored_since=2015-11-05T15%3A47%3A32.740000%2B00%3A00&role=observation&cone=%5B229.2874%2C+35.7445%2C+0.5%5D&authored_until=2016-01-04T15%3A47%3A32.740000%2B00%3A00 HTTP/1.1" 200 525
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): voeventdb.4pisky.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /apiv1/list/ivorn?authored_until=2016-01-04T15%3A47%3A32.740000%2B00%3A00&authored_since=2015-11-05T15%3A47%3A32.740000%2B00%3A00&limit=32&cone=%5B229.2874%2C+35.7445%2C+0.5%5D&offset=0&role=observation HTTP/1.1" 200 2502
Out[16]:
[u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Known_Pos_1745116191-419',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-562',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-563',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-599',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-625',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-633',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_QuickLook_Pos_666352-546',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745115961-415',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745116036-416',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1745116081-418',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_666352-551',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_17443568-112',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_666352-552',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-574',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-611',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos_666352-636',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-578',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-612',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-570',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-604',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-569',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-600',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image_666352-554',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve_666352-561',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image_666352-555',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-560',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-568',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-559',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-564']

As a result of restricting our search to a window about the XRT-position timestamp, we've cut out all of those 'SubSubThresh' events that probably weren't related.

A note of warning

What hasn't been mentioned so far is that a query with a cone-search filter will only return VOEvents which contain a position in the WhereWhen section of the packet. This sounds obvious, but on the other hand it's worth being reminded that some related and interesting VOEvents may not contain any position information.

Most packets relating to follow-up or simultaneous observations of an event will contain position data, but we can imagine circumstances where this wouldn't apply, e.g. an all-sky high-energy gamma-ray detector without good localization. Alternatively a VOEvent author may simply decide not to repeat co-ordinates that have been given in an earlier packet (perhaps when giving refined estimates about flux / lightcurve / spectral index, etc), but instead rely on the citation mechanism - see the next notebook!

Coming next ...

In the search above for packets near to a Swift XRT position, many of the packets were clearly designated as relating to the same event - they all had the same ID number. Is that relation encoded into the packet-data? Can we just select the VOEvents which are already marked as being related to a particular packet?

The answer, of course, is yes - we make use of citation data.

Tutorial 4

Right click here and 'save as' to download this notebook.

Exploring citation networks

In the previous notebook, we retrieved VOEvents related to a Swift XRT detection using the cone-search filter. A cone-search can bring to light new associations between VOEvents from different observatories, but we saw that it can also return unrelated events that just happen to lie nearby.

In this notebook, we'll go about exploring relations between VOEvents which are already encoded by the packet author, using the citation mechanism.

In [1]:
from __future__ import print_function
In [2]:
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1
from voeventdb.remote.apiv1 import FilterKeys
from voeventdb.remote.helpers import Synopsis

Once again, we'll retrieve a handy example VOEvent representing a Swift XRT detection of a GRB:

In [3]:
xrt_synopsis = Synopsis(apiv1.packet_synopsis('ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553'))

References and citations

Note on terminology:

In voeventdb, we use the terms 'reference' and 'citation' in their precise bibliographic sense, i.e. references are made (to another packet), and citations are received (from other packets).

References made by a packet

It's easy to inspect the references made by a packet - it's right there in the synopsis:

In [4]:
print("This packet contains {} references.".format(len(xrt_synopsis.references)))
xrt_synopsis.references[0]
This packet contains 1 references.
Out[4]:
{u'cite_type': u'followup',
 u'description': u'This is the XRT Position for the original BAT trigger.',
 u'ref_ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547'}

Citations received

We can also check if this packet receives any citations. To do so, we search for other VOEvents which list it as a reference:

In [5]:
my_filters = {FilterKeys.ref_exact:xrt_synopsis.ivorn}
apiv1.list_ivorn(my_filters)
Out[5]:
[]

Nope! No citations. But what about the original BAT trigger, the one referenced by the XRT VOEvent?

In [6]:
my_filters = {FilterKeys.ref_exact:xrt_synopsis.references[0]['ref_ivorn']}
citations_to_bat = apiv1.list_ivorn(my_filters)
citations_to_bat
Out[6]:
[u'ivo://voevent.phys.soton.ac.uk/AMI-REQUEST#151205-1548.10',
 u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_666352-551',
 u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_666352-552',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image_666352-554',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image_666352-555',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-559',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-560',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve_666352-561',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-562',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-563',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-564',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-568',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-569',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-570',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-574',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-578',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Scaledmap_666352-595',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-599',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-600',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-604',
 u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-611',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-612',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-625',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-633',
 u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos_666352-636']

Number of citations received, batch version:

Aha! So, the original BAT GRB trigger is the 'anchor reference' for all these other packets. Are any of them cited, in turn? There's a quick way to find out; we can use the ivorn_cited_count endpoint to get a citation count for all the packets matching our current filter set (for this example, we'll re-use the 'exact reference' filter):

In [7]:
my_filters
Out[7]:
{'ref_exact': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547'}
In [8]:
apiv1.list_ivorn_ncites(my_filters)
Out[8]:
[[u'ivo://voevent.phys.soton.ac.uk/AMI-REQUEST#151205-1548.10', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_666352-551', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_666352-552', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image_666352-554', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image_666352-555', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-559', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-560', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve_666352-561', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-562', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-563', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-564', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-568', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-569', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-570', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-574', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-578', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Scaledmap_666352-595', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-599', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-600', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-604', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-611', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-612', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-625', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-633', 0],
 [u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos_666352-636', 0]]

Again, a big fat nope - they all have zero citations.

Party trick - network mapping

So we have quite a boring citation network - several packets cite the BAT GRB position, then the trail ends. Nonetheless, we can use it to show off a party trick - voeventdb.remote contains an extra function that makes repeated calls to the server to perform a depth-first search (with configurable maximum recursion level) of the citation network:

In [9]:
cite_map = apiv1.citation_network_map(xrt_synopsis.ivorn)
In [10]:
cite_map
Out[10]:
{u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547': {u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-562',
  u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-563',
  u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-599',
  u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-625',
  u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Lightcurve_666352-633',
  u'ivo://nasa.gsfc.gcn/SWIFT#BAT_Scaledmap_666352-595',
  u'ivo://nasa.gsfc.gcn/SWIFT#FOM_Obs_666352-551',
  u'ivo://nasa.gsfc.gcn/SWIFT#SC_Slew_666352-552',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-574',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Image_666352-611',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Nack_Pos_666352-636',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-578',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_Image_666352-612',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-570',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_Proc_SrcList_666352-604',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-569',
  u'ivo://nasa.gsfc.gcn/SWIFT#UVOT_SrcList_666352-600',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Image_666352-554',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Lightcurve_666352-561',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-553',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Pos_666352-605',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Image_666352-555',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-560',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_666352-568',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-559',
  u'ivo://nasa.gsfc.gcn/SWIFT#XRT_Spec_666352-564',
  u'ivo://voevent.phys.soton.ac.uk/AMI-REQUEST#151205-1548.10'}}

If we use matplotlib and networkx, we can even draw the citation map. You can see the 'BAT_GRB' packet with many edges, representing citations, leading out from it

(To do: More elaborate plots, with readable labels / colouring, etc. Contributions welcome!)

In [11]:
%matplotlib inline
import networkx as nx
G=nx.DiGraph()
for ivorn in cite_map.keys():
    G.add_node(ivorn)
for origin_ivorn, citing_ivorns in cite_map.items():
    for ci in citing_ivorns:
        G.add_edge(ivorn,ci)
nx.draw_networkx(G, arrows=True, with_labels=False)

Tutorial 5

Right click here and 'save as' to download this notebook.

Surveying recent events and planning your followup

We've now covered all the endpoints provided by voeventdb. In this notebook, we'll look at how to request particular list-orderings, and demonstrate how you might use voeventdb to plan a night's follow-up observing.

In [1]:
from __future__ import print_function
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1
from voeventdb.remote.apiv1 import FilterKeys
from voeventdb.remote.helpers import Synopsis
from datetime import datetime, timedelta
import pytz

List ordering

When using the voeventdb list-query endpoints, we can control ordering using the order parameter. So for example, if we simply request the first 50 IVORNs with no modifiers, we get the first entries inserted into the database:

In [2]:
apiv1.list_ivorn(n_max=5)
Out[2]:
[u'ivo://nasa.gsfc.gcn/Fermi#GBM_Alert_2015-11-19T17:05:04.87_469645508_1-285',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_46-286',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_57-287',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_68-289',
 u'ivo://nasa.gsfc.gcn/Fermi#GBM_Flt_Pos_2015-11-19T17:05:04.87_469645508_72-293']

In this case, whatever happened to arrive just after the database was switched on, before the previous year's events were loaded from an archive. But, we can change the ordering, choosing from one the order-values. For example, to retrieve the most recent 'observation' VOEvents, we'll request the list in 'author-datetime descending' order:

In [3]:
apiv1.list_ivorn(filters={FilterKeys.role:'observation'},
                 order=apiv1.OrderValues.author_datetime_desc,
                 n_max=5, 
                )
Out[3]:
[u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1748898606-019',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1748898601-018',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1748898286-017',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1748898286-016',
 u'ivo://nasa.gsfc.gcn/SWIFT#BAT_SubSubThresh_Pos_1748897676-015']

Retrieving recent GRB events

With this last feature, we can start using voeventdb for applications such as reviewing the most recent alerts, and perhaps even planning our follow-up. Suppose we want to retrieve the 10 most-recent GRB-event alerts from the Swift BAT:

In [4]:
filters = { FilterKeys.ivorn_contains: 'BAT_GRB',
            FilterKeys.role: 'observation'}
now = datetime.utcnow()
recent_swift_grb_ivorns = apiv1.list_ivorn(filters,
                                     order=apiv1.OrderValues.author_datetime_desc,
                                     n_max=10,
                                    )
recent_swift_grbs = [Synopsis(apiv1.packet_synopsis(i)) for i in recent_swift_grb_ivorns]

Let's view a summary table, displaying just the dates and co-ords:

In [5]:
print("Recent GRBs as of {}:".format(now))
print()
for grb in recent_swift_grbs:
    print(grb.author_datetime, grb.coords[0].ra.deg, grb.coords[0].dec.deg)
Recent GRBs as of 2015-12-14 15:27:44.132240:

2015-12-10 03:14:27+00:00 65.1823 -71.2304
2015-12-09 06:23:03+00:00 282.0884 -2.4214
2015-12-05 21:43:43+00:00 41.1585 -43.4806
2015-12-05 15:48:06+00:00 229.2823 35.7691
2015-11-27 09:09:08+00:00 19.7778 -82.7738
2015-11-22 12:09:49+00:00 274.7419 -15.9781
2015-11-22 08:50:21+00:00 184.205 17.5986
2015-11-18 03:06:47+00:00 57.1581 65.8686
2015-11-14 09:59:48+00:00 120.9374 -61.0422
2015-11-12 13:45:00+00:00 2.0452 -61.6924

If we wanted to be a little fancy, we could use a pandas dataframe to draw the table for us, applying some formatting tricks along the way:

In [6]:
import pandas as pd
In [7]:
df = pd.DataFrame(index = [g.author_datetime for g in recent_swift_grbs],
                  data = {'ra': [g.coords[0].ra.deg for g in recent_swift_grbs],
                          'dec': [g.coords[0].dec.deg for g in recent_swift_grbs],
    })
#Re-order the columns
df = df[['ra','dec']]
#Set the float-display formatting:
pd.options.display.float_format = '{:+07.2f}'.format
print("Recent GRBs as of {}:".format(now))
df
Recent GRBs as of 2015-12-14 15:27:44.132240:
Out[7]:
ra dec
2015-12-10 03:14:27 +065.18 -071.23
2015-12-09 06:23:03 +282.09 -002.42
2015-12-05 21:43:43 +041.16 -043.48
2015-12-05 15:48:06 +229.28 +035.77
2015-11-27 09:09:08 +019.78 -082.77
2015-11-22 12:09:49 +274.74 -015.98
2015-11-22 08:50:21 +184.21 +017.60
2015-11-18 03:06:47 +057.16 +065.87
2015-11-14 09:59:48 +120.94 -061.04
2015-11-12 13:45:00 +002.05 -061.69

10 rows × 2 columns

Wrapping up

That about covers all the major features of voeventdb, as demonstrated using the voeventdb.remote client-library. Comments, questions, bug-reports and other contributions are all welcomed - you can leave a note on the issue tracker or find more contact details at http://4pisky.org/voevents/.

And finally...

We've covered how to get data out of voeventdb - but not what to do with the data when you've got it. The examples notebook demonstrates a few basic ideas - see next!

Demo applications and data-visualizations

Right click here and 'save as' to download this notebook.

Data-analysis and visualization examples

This notebook does not directly demonstrate voeventdb (see the tutorials!) but provides examples of what's possible when data on astronomical transients is readily available:

Prelude - fetch 10 most-recent GRBs

In [1]:
from __future__ import print_function
import voeventdb.remote as vr
import voeventdb.remote.apiv1 as apiv1
from voeventdb.remote.apiv1 import FilterKeys
from voeventdb.remote.helpers import Synopsis
from datetime import datetime, timedelta
import pytz
In [2]:
filters = { FilterKeys.ivorn_contains: 'BAT_GRB',
            FilterKeys.role: 'observation'}

recent_swift_grb_ivorns = apiv1.list_ivorn(filters,
                                     order=apiv1.OrderValues.author_datetime_desc,
                                     n_max=10,
                                    )
recent_swift_grbs = [Synopsis(apiv1.packet_synopsis(i)) for i in recent_swift_grb_ivorns]

Plotting a timeline

In the last tutorial we retrieved and printed the timestamps of the most recent GRBS - but deciphering textual timestamps isn't much fun. Let's plot a timeline instead:

In [3]:
%matplotlib inline
import matplotlib as mpl
import matplotlib.pyplot as plt 

grb_dtimes=[s.author_datetime for s in recent_swift_grbs]

fonts = {'weight' : 'bold',
        'size'   : 14}

mpl.rc('font', **fonts)

now = pytz.UTC.localize((datetime.utcnow()))
week_markers = [now - timedelta(days=7)*w for w in range(0,5)]


markersize=95

plt.scatter(grb_dtimes, [1]*len(grb_dtimes),  marker='*', s=markersize, label='GRB')
plt.scatter(now, 1, marker='o', s=markersize, c='r')
for d in grb_dtimes:
    plt.axvline(d, ymax=0.5, ls='-')

first_label = True
for w in week_markers:
    plt.axvline(w, ymax=0.5, ls='--',c='r', 
                label=('Week marker' if first_label else None))
    first_label=False

plt.xlim(min(grb_dtimes)-timedelta(days=2), max(max(grb_dtimes),now)+timedelta(days=2))
plt.gcf().autofmt_xdate()
ax = plt.gca()
ax.yaxis.set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['left'].set_visible(False)
ax.spines['top'].set_visible(False)
ax.xaxis.set_ticks_position('bottom')
ax.get_yaxis().set_ticklabels([])
plt.legend(loc='best')
plt.gcf().set_size_inches(12,4)
plt.gcf().suptitle("Recent Swift BAT alerts, as of {}".format(now), fontsize=22)
Out[3]:
<matplotlib.text.Text at 0x7f279b169550>

Creating a sky-coordinate scatterplot

Let's see where those GRBs lie on-sky (with credit to http://www.astropy.org/astropy-tutorials/plot-catalog.html):

In [4]:
from astropy.coordinates import Angle
import astropy.units as u
grb_ra_coords = Angle([grb.coords[0].ra for grb in recent_swift_grbs])
grb_dec_coords = Angle([grb.coords[0].dec for grb in recent_swift_grbs])
grb_ra_coords = grb_ra_coords.wrap_at(180*u.degree)

print(grb_ra_coords.deg)
print(grb_dec_coords.deg)
[  65.1823  -77.9116   41.1585 -130.7177   19.7778  -85.2581 -175.795
   57.1581  120.9374    2.0452]
[-71.2304  -2.4214 -43.4806  35.7691 -82.7738 -15.9781  17.5986  65.8686
 -61.0422 -61.6924]
In [5]:
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111, projection="mollweide")
ax.scatter(grb_ra_coords.radian, grb_dec_coords.radian, marker='*', s=13**2, label='GRB')
ax.grid(True)
plt.legend()
plt.gcf().suptitle('Locations of 10 most recent Swift BAT GRB alerts', fontsize=24)
Out[5]:
<matplotlib.text.Text at 0x7f279b062350>
/usr/lib/pymodules/python2.7/matplotlib/projections/geo.py:485: RuntimeWarning: invalid value encountered in arcsin
  theta = np.arcsin(y / np.sqrt(2))

Planning an observation with astropy

Let's suppose we want to observe the a recently alerted GRB event from Cambridge, UK. We have the location of the event, already converted to astropy co-ordinates. This means we can use astropy's Alt-Az conversions to calculate the altitude of the target at any given time.

(This section borrows heavily from http://docs.astropy.org/en/stable/coordinates/observing-example.html, consult the original for more detail and background)

In [6]:
#Grab the latest IERS time-data first:
from astropy.utils.data import download_file
from astropy.utils import iers
iers.IERS.iers_table = iers.IERS_A.open(download_file(iers.IERS_A_URL, cache=True))
In [7]:
import numpy as np
from astropy import units as u
from astropy.time import Time
from astropy.coordinates import SkyCoord, EarthLocation, AltAz

Let's pick a GRB which will actually be observable some of time, i.e. one in the same hemisphere as Cambridge:

In [8]:
accessible_grbs = [g for g in recent_swift_grbs if g.coords[0].dec.deg>0]
len(accessible_grbs)
Out[8]:
3
In [9]:
grb_target = accessible_grbs[0]
print("Sample target GRB:")
print(grb_target)
Sample target GRB:
{ 'author_datetime': '2015-12-05T15:48:06+00:00',
  'author_ivorn': u'ivo://nasa.gsfc.tan/gcn',
  'ivorn': u'ivo://nasa.gsfc.gcn/SWIFT#BAT_GRB_Pos_666352-547',
  'received': '2015-12-05T15:48:09.180095+00:00',
  'references': [],
  'role': u'observation',
  'sky_events': [ <SkyCoord (ICRS): (ra, dec) in deg
    (229.2823, 35.7691)> +/- 0.05 @ 2015-12-05T15:46:00.930000+00:00],
  'stream': u'nasa.gsfc.gcn/SWIFT',
  'version': u'2.0'}
In [10]:
grb_target_posn = accessible_grbs[0].coords[0]
cambridge = EarthLocation(lat=52.205*u.deg, lon=0.118*u.deg, height=6*u.m)
utcoffset = -0*u.hour  # GMT
time = Time.now() - utcoffset
grb_altaz = grb_target_posn.transform_to(AltAz(obstime=time,location=cambridge))  
print("GRB's current altitude = {0.alt:.2}".format(grb_altaz)  )
GRB's current altitude = 3e+01 deg
In [11]:
from astropy.coordinates import get_sun
now = Time.now()
delta_24 = np.linspace(0, 24, 100)*u.hour
times = now + delta_24

altaz_frame = AltAz(obstime=times, location=cambridge)

grb_altaz_array = grb_altaz.transform_to(altaz_frame)  
sun_altaz_array = get_sun(times).transform_to(altaz_frame)
In [12]:
fig = plt.figure(figsize=(12,8))
plt.plot(delta_24, grb_altaz_array.alt, lw=10, label='GRB')  
plt.plot(delta_24, sun_altaz_array.alt, lw=10, c='y', label='Sun')  
# plt.fill_between(delta_midnight, 0, 90, sun_altaz_array.alt < -0*u.deg, color='0.5', zorder=0)  
# plt.fill_between(delta_midnight, 0, 90, sun_altaz_array.alt < -18*u.deg, color='k', zorder=0)  

plt.axhline(0, ls='--', lw=8, c='k',
           label='Horizon',)
plt.xlabel('Hours from {}'.format(now))  
plt.ylabel('Source altitude')
plt.legend(loc='best')
plt.gcf().suptitle("GRB target altitude over next 24 hours", fontsize=24)
Out[12]:
<matplotlib.text.Text at 0x7f279ad56490>

API Reference

Package default variables

Define some package-level default values:

voeventdb.remote.default_host = 'http://voeventdb.4pisky.org'

The default host to query.

voeventdb.remote.default_list_n_max = 5000

Maximum number of rows to fetch from a list request, by default.

If you really want to fetch more rows in a single query, this can be changed for a single request by setting the n_max parameter-value. Setting a value of n_max=0 will fetch all available rows.

voeventdb.remote.default_pagesize = 1000

Number of rows fetched in each HTTP GET request (when querying list-endpoints).

Note that if more rows are available, multiple GET requests are made ‘behind the scenes’ when using the top-level interface, so this variable can typically be left unchanged for casual usage - it’s aimed at advanced usage when trying to improve performance with large queries.

API version 1 (apiv1)

class voeventdb.remote.apiv1.FilterKeys[source]

Enumerates valid filters.

Useful for building dictionaries representing filter-sets, eg.:

filters = {
    FilterKeys.ivorn_contains: 'GRB',
    FilterKeys.role : 'observation',
}

For definitions of the various filters, and examples of valid values, see the voeventdb query-filters page.

authored_since = 'authored_since'
authored_until = 'authored_until'
cited = 'cited'
cone = 'cone'
coord = 'coord'
ivorn_contains = 'ivorn_contains'
ivorn_prefix = 'ivorn_prefix'
ref_any = 'ref_any'
ref_contains = 'ref_contains'
ref_exact = 'ref_exact'
role = 'role'
stream = 'stream'
class voeventdb.remote.apiv1.OrderValues[source]

Enumerates valid orderings.

For details see this section in the voeventdb.server docs.

author_datetime = 'author_datetime'
author_datetime_desc = '-author_datetime'
id = 'id'
id_desc = '-id'
ivorn = 'ivorn'
ivorn_desc = '-ivorn'
voeventdb.remote.apiv1.convenience_funcs.map_authored_month_count(filters=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.citation_network_map(ivorn, max_recursion_levels=5)[source]
voeventdb.remote.apiv1.convenience_funcs.count(filters=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.list_ivorn(filters=None, order=None, pagesize=None, n_max=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.list_ivorn_ncites(filters=None, order=None, pagesize=None, n_max=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.list_ivorn_nrefs(filters=None, order=None, pagesize=None, n_max=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.map_role_count(filters=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.map_stream_count(filters=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.map_stream_role_count(filters=None, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.packet_synopsis(ivorn, host=None)[source]
voeventdb.remote.apiv1.convenience_funcs.packet_xml(ivorn, host=None)[source]

Helper classes

Helper classes for providing convenience parsing of returned JSON content.

class voeventdb.remote.helpers.SkyEvent(skyevent_dict)[source]

Represents universal attributes of an observed event on sky.

I.e. the most basic details that we expect to find in all packets reporting events with sky-positions.

position

astropy.coordinates.SkyCoord

Best-estimate sky-coordinates of the event being reported.

position error

astropy.coordinates.Angle

Error-cone on the position estimate.

timestamp of event

datetime.datetime

Timestamp for the reported event (UTC timezone).

class voeventdb.remote.helpers.Synopsis(synopsis_dict, api_version_string='apiv1')[source]

Parses the output from the ‘synopsis’ endpoint into a class-object.

author_datetime

datetime.datetime

The Who.Date timestamp. (Parsed to datetime, UTC timezone). May be None if no timestamp present in the VOEvent packet.

author_ivorn

string

The Who.AuthorIVORN entry. May be None if no entry present in the VOEvent packet.

ivorn(string)
received

datetime.datetime

Dates exactly when the packet was loaded into this instance of voeventdb.

role(string)
stream(string)
version(string)
sky_events

list

A list of SkyEvent objects. These are parsed from the WhereWhen section of a VOEvent packet, if possible. (If the packet has no WhereWhen data, this is an empty list).

coords

list

astropy.coordinates.SkyCoord positions. This is a syntax-shortcut, the positions are just those of the sky_events entries, if any are present.

references

list

References to other VOEvent packets. This is a list of dictionaries representing any references present in the Citations section of the VOEvent packet.

Indices and tables