VO Desktop2008beta-doc > Scripting > Python

Scripting With Python

You can access the services and tools provided by Astrogrid from several scripting/programming languages. In this document we describe how this is done from Python. Python is a high level programming language which is becoming very popular in astronomy.

Requirements

Python

In order to access the Astrogrid services from Python you need a recent version of Python installed. Python versions 2.4 (released on November 30, 2004) and greater come with all needed libraries but the latest version of Python is recommended (2.5 released on September 19th, 2006). In order to know which python version you are running you can type from the command line::

python -c 'import sys; print sys.version'

If you do not have a version greater or equal to 2.4 then you will need to install a more recent Python. Find below some instructions on installing Python in different system. More details are available in the Beginners Guide Downloads Section http://wiki.python.org/moin/BeginnersGuide/Download.

If you are running Windows just go to the Python download page (http://www.python.org/download/) and choose Python 2.5 Windows Installer. By default Python will be installed in the directory C:\Python25. However, it is recommended that new users install using ez_setup.py as described under 'Installation Using Setuptools'.

If you use Linux then Python is included in all main distributions by default. Refer to your package install instructions to install it if it is not already in your system. Generally something like apt-get install python or yum install python will be enough.

Mac OS X (10.4) comes with Python 2.3 installed, but you will need to use Python 2.4 or later. There are several ways of getting it:

• Fink Project (http://finkproject.org/). This is probably the best if you are used to Linux. Together with Python and all additional packages you can install a whole load of unix tools.
• PythonMac (http://pythonmac.org/packages/)
• You can also follow instructions in here: http://www.physics.usyd.edu.au/astrop/ausvoss/index.php/Main/InstallMacOSX

OSX 10.5 already comes with Python 2.5 installed so you do not need to do anything else additional.

Astrogrid VO Desktop

Download from http://www.astrogrid.org

Installation and Configuration

Quick installation

For a quick installation which does not involve any third party packages follow the instructions below. Note however that if you plan to use Python seriously you should consider installing packages like numpy (for numerical processing), pyfits/pcfitsio (to read fits files), matplotlib (to create plots) and others and then the setuptools route discussed below is more appropriate. Also updates are handled easier. Anyway to quickly start with Python and Astrogrid:

• Download the latest egg file from http://code.google.com/p/pyacr and save it somewhere in your disk (e.g. $HOME/astrogrid.egg)
• Set the PYTHONPATH enviromental variable to include the downloaded file (e.g. export PYTHONPATH=$HOME/astrogrid.egg). Windows users see notes under Installation Using Setuptools.

Then go to the configuration section directly.

Installation using setuptools

If you do not have setuptools already installed then download the latest ez_setup.py http://peak.telecommunity.com/dist/ez_setup.py and run it (double click on Windows); this will download and install the appropriate setuptools egg for your Python version.

Then (for linux, MAC OS X or Windows DOS command line) you just need to type::

easy_install -Uf http://code.google.com/p/pyacr/downloads/list astrogrid

For more information using easy_install look at the documentation http://peak.telecommunity.com/DevCenter/EasyInstall and in custom locations http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations if you do not have administrator access to the Python installation directory.

For Linux/Ubuntu, install with

sudo apt-get install python-setuptools

For Windows, after running "ez_setup.py" via double click you need to add python & scripts folder to your PATH variable, e.g. in Vista:

Start - (right click) Computer - Properties - Advanced System Settings - Environment Variables

Then in System Variables window, scroll down to PATH and add

";C:\Python25\Scripts;C:\Python25"

This method is also described at http://cybertechhelp.com/forums/blog.php?b=4

Configuration

This module expects a configuration file located $HOME/.python-acr (or $HOME/_python-acr for Windows Users). In order to create one just import the astrogrid module from python, it will automatically detect that you do not have one and will create a skeleton one for you::

   python -c 'import astrogrid'

This default configuration file looks like::

  debug = True
  verbose = True
  autologin = True
  plastic = True
  
  [community]
  default=leicester
  
  [[leicester]]
  username = username1
  password = password1
  community = uk.ac.le.star
  
  [[ukidss]]
  username = username2
  password = password2
  community = ukidss.roe.ac.uk

where username[1,2] and password[1,2] are your particular credentials for logging into your AstroGrid accounts. With this configuration you get maximum verbosity (debug, verbose) when running your applications. Also you will not be asked about your login credentials since these will be read from the configuration file when needed. In the second example you will be logged into in your ukidss account by default but you can select the leicester account as well.

Make sure that the configuration file has right properties, i.e. it is only readable for the owner. In Unix you do chmod 0600 ~/.python-acr.

In Windows, edit user, pass as above for AstroGrid and UKIDSS from DOS using, e.g.

edit _python-acr

Encrypting the configuration file

For an extra layer of security you have the option to encrypt your configuration file. Of course this means that you will be prompted for the password used to encrypt it when you use acr. To do this, inside python::

from astrogrid.config import cryptconf
cryptconf()

and type a password. The same command will decrypt a previously encrypted file.

Getting Started

In order to use AR from Python you need the AstroGrid VO Desktop running. Then you need to type the following:

$ python
>>> from astrogrid import acr

and in Windows from the DOS command line:

> python
>> from astrogrid import acr

Various DEBUG warning flags may appear by default but can be switched off if preferred.

How To…

Log In and Out

Once you have a configuration file as described above you can login into your different accounts using::

# Login in the default account
acr.login()

# Login in your ukidss account
acr.login('ukidss')

# Login user particular credentials
acr.login(username, password, community)

To log out simply type::

acr.logout()

Getting help

Suppose that we want to perform a conesearch, so we first import the ConeSearch? class::

from astrogrid import ConeSearch

You can get help about the usage of the class and its methods by typing::

help(ConeSearch)

Query the Registry

The registry holds the information about all services available within the VO. Below some examples on querying the registry for resources.

from astrogrid import Registry
reg = Registry()

# search for Cone services related to SDSS
# and print their ids
list = reg.searchCone('SDSS')
print [p['id'] for p in list]

# Print the description of service number 5 of the list
print list[5]['content']['description']

Perform a SIAP Query

This are the basic lines to perform a SIAP search. For a more complete example and link to a full working script look below in the examples section::

from astrogrid import SiapSearch
siap = SiapSearch('ivo://roe.ac.uk/services/SIAPDR4-images')
result = siap.execute(180.0, 2.0, 1.0)  

Query a catalogue using ADQL

In order to query a service which provides ADQL capabilities we use the Data Set Access class (DSA). The following example illustrates the basic commands to query the 2MASS PSC catalogue:

from astrogrid import DSA
db = DSA('ivo://wfau.roe.ac.uk/twomass-dsa/ceaApplication')
app = db.query('SELECT TOP 10 * FROM twomass_psc AS x')

app.status()

result = app.results()[0]

Work with !VOSpace

This is how you list the contents of your root directory of !VOSpace::

from astrogrid import MySpace
m = MySpace()
m.ls()

You can delete files or folders:

m.rm('#test/file.vot')
m.rm('#test/', recursive=True)

and read file from !VOSpace::

img = m.readfile('#sdss/image.fits')

File names in !VOSpace always start with the hash key (#) or with 'ivo://'. For instance when broadcasting a file to plastic listening applications::

# A file in local disk
broadcast('image.fits')

# A file in VOSpace
broadcast('#image.fits')

Send results to TOPCAT or Aladin

If TOPCAT or Aladin (or any other Plastic listening application) are running t hen you can send the results from your queries or tasks directly to them. This is how::

# Start plastic connection
acr.startplastic()
   
# Send a local file to Aladin
acr.plastic.broadcast('image.fits', 'Aladin')

# Send a file in VOSpace to all applications
acr.plastic.broadcast('#sdss/catalogue.vot')

# Send the result from a cone search to Topcat
result = cone.execute(250.0, 54.0, 0.3)
acr.plastic.broadcast(result, 'Topcat')   

Run processes in the background

Using IPython it is possible to run any process in the background. E.g.::

cone = ConeSearch("ivo://ned.ipac/Basic_Data_Near_Position")
%bg cone.execute(180.0, 1.0, 0.1)
%bg cone.execute(180.0, 2.0, 0.1)
%bg cone.execute(180.0, 3.0, 1)

jobs.status()
Running jobs:
2 : cone.execute(180.0, 3.0, 1)

Completed jobs:
0 : cone.execute(180.0, 1.0, 0.1)
1 : cone.execute(180.0, 2.0, 0.1)

jobs[1].status()
'Completed'

votable = jobs[1].result

Examples

Below some examples which how some of the capabilities of the system.

SIAP Image Search

The Simple Image Access Protocol allows to query a service for images which have observed a particular area of sky. The following example reads a list of object names from a file and uses the Sesame service to look at the coordinates of the object. Then for each of them, queries the SDSS DR4 image service for images which have observed the object. The resultant votables are saved to a file for each object in the local disk.

# We read a list of object names from a file   
# (see the complete script)

# Initialize siap service search and name resolver
siap = SiapSearch('ivo://roe.ac.uk/services/SIAPDR4-images')
s = sesame()

# Loop for each object of the list, query Vizier to get coordinates and execute
# the SIAP search. The resultant VOTable is then saved to a file in the local
# directory named after the object.
for obj in objects:
   coords, ra, dec = s.resolve(obj)
   votable = siap.execute(ra, dec, 30.0/3600.0)
   open('%s.vot' % obj, 'w').write(votable)

One can substitute the last loop for the following one in order to save the output tables to !VOSpace instead of local disk:

   # Same loop as above. Now we save the VOTables to VOSpace
   # We convert spaces to underscores in the saved filename due to VOSpace complains
   # We do not overwrite existing files. We assume here that the username, password
   # and community credentials are specified in the configuration file (see docs)
   for obj in objects:
        coords, ra, dec = s.resolve(obj)
        ofile = '#siap/%s.vot' % obj.replace(' ','_')
        votable = siap.execute(ra, dec, 30.0/3600.0, saveAs=ofile)

Note that a cone search is done exactly in the same way using ConeSearch instead of SiapSearch.

Extracting objects from an image

In this example we are going to show how to fill in the necessary parameters needed to run a application and then submit it to the system.

In this example the variables params, filter and config are strings containing the SExtractor configuration files (i.e. they are not the names of the files but the contents o the files) so it is not necesary to set the 'indirect' flag.

# Application ID. This is the name of the application we are going to run.
# It can be obtained from a registry search.
id = 'ivo://org.astrogrid/SExtractor'

# We fill in the application parameters
app = Applications(id)

app.inputs['ANALYSIS_THRESH']['value']=1.5
app.inputs['IMAGE_BAND']['value']='R'
app.inputs['MAG_ZEROPOINT']['value']=25.0
app.inputs['SEEING_FWHM']['value']=1.2

app.inputs['PARAMETERS_NAME']['value']=params
app.inputs['FILTER_NAME']['value']=filter
app.inputs['config_file']['value']=config

app.inputs['DetectionImage']['value'] = '#sextractor/image.fits'
app.inputs['PhotoImage']['value'] = '#sextractor/image.fits'

app.outputs['CATALOG_NAME']['value'] = '#sextractor/image_cat.fits'

# We submit the application.
task=app.submit()

# We can poll the status of the application
task.status()

It is possible to send the image and generated catalogue to Aladin::

from astrogrid.plastic import broadcast
broadcast('#sextractor/image.fits')
broadcast('#sextractor/image_cat.fits')

The result is shown in the figure below:

Submitting feedback

Your feedback is welcome. To submit your comments from within python you just need to type::

from astrogrid import feedback
feedback.submit()

An editor will appear (it depends on your EDITOR environment variable or uses vi or notepad as defaults) in which you can write your feedback. Close the editor to finish the submission. Please use this only for comments on the Python wrapper to the ACR not for general astrogrid comments.

Example Python Scripts

Python scripts are available as a tar file in here: python.tar.gz