Programmatic Interface - Overview

Topics on this Page


Overview

This page describes the Programmatic interface which is a method of remotely controlling a NetRS receiver using easily generated text-based commands. This interface exists in parallel to the Graphical User Interface (GUI) that is accessible through the HTTP and HTTPS ports of the receiver. The GUI interface uses HTML web pages to provide visual, point-and-click access to all information and controls on the system. It is designed to be interactive with the user, and is not well suited for automating remote control tasks. The Programmatic Interface provides a matching functionality using a more machine friendly method. It accepts easily generated, text-based commands as requests to the system. It responds with simple, plain-text answers that are easily interpretted by an automatic process.

Programmatic commands are submitted to the NetRS using the same TCP/IP channels used by the GUI-browser interface. They both use the HTTP or HTTPS protocol to submit CGI requests encoded in URLs. Response documents are then returned to the remote computer using the same channels. The only difference between GUI requests and Programmatic requests is the exact URLs used, and the types of responses returned.

Secure operations using the Programmatic interface is available in two ways. First, the interface can be accessed through the HTTPS protocol, which encrypts all transactions. This prevents anyone from observing the interactions. Note that the commands can be accessed over the unencrypted HTTP port as well, which is not encrypted.

The second level of security is provided by the HTTP account mechanism. All programmatic commands are handled through the GUI web server inside the NetRS, and are restricted to users with Sysadmin privileges and passwords. Note that this level of security is NOT enabled by default. You must enable the sysadmin operation restrictions before accounts and passwords will be required for the Programmatic Interface. Also note that this security level, when activated, applies to ALL programmatic commands, regardless of which access levels are required for the corresponding Browser-GUI command. In other words, executing any Programmatic Interface command will require sysadmin access permissions.

Format of Programmatic Commands

Programmatic commands are formatted as if they were URL CGI requests. A typical request would look like:
    http://RS0000017332/prog/enable?session&name=testing
http: The protocol to use, either 'http:' or 'https:'.
//RS0000017332 The name or IP address of the NetRS unit that is the target of the command.
Must be preceded by two slashes, '//'.
/prog Indicator to the NetRS that this is a programmatic request. This triggers the command parser to interpret the command.
/enable The Verb. This is the action requested. Verbs can be one of:
  • Show
  • Set
  • Reset
  • Enable
  • Disable
  • Delete
  • Create
  • Upload
  • Download
  • Verify
  • Install
In this sample, the Verb is 'enable'.
?session This is an Object type on which the action is to be performed. Objects can be:
  • A state or characteristic of the system, such as the Serial Number.
  • A function control, such as ElevationMask or SystemName.
  • A file object, like a logged data file, or a firmware file.
In this sample, the Object is a data logging session.
&name=testing A parameter that picks one of several objects, or describes characteristics that qualify the command request.
In this sample, the 'name' parameter gives the name of the specific data logging session that is to be enabled.
Parameters are usually of the form paramname=value. Multiple parameters can be given for a single command. They all must be separated by ampersand characters, ' & '.

The general command format format is:

    http://SystemName/prog/Verb?Object&param=value&param=value...

In most of this document, parts of that format will be left out for clarity. So commands will be shown as

    Verb  Object  param=value  param=value ...

As far as is possible, Verbs, Objects, parameter names, and parameter values are Case Insensitive. That is, you can use any combination of upper and lower case characters, anywhere in the commands. The exceptions are items like SystemName, file names, session names, account names, etc. These exceptions are called out in the parameter descriptions for each command.

Command parameters are of the form paramname=value. The value is a string of characters, usually a number or a single word. A few parameters contain multiple items concatenated into a single string, often with punctuation. For example, some parameters allow a comma-separated list like this:

    Set GpsSatControls disable=5,8,23
In such cases, the parameter listing in the command description will show the acceptable formats.

Some parameters define an arbitrary string of characters which can include spaces, ampersands or other punctuation characters. These characters may cause problems when encoded in URLs. It is best to express them as "%xx" values, where 'xx' is the hexadecimal equivalent of the ASCII character. Common values that need encoding are:
%20Space
%25Percent, %
%26Ampersand, &
%3FQuestionMark, ?
Most other characters will not require encoding, but it cannot hurt to encode them if there is any doubt.

Uploading Files

A small number of commands are used to transfer a file from an external computer into the file system of the NetRS system. These commands all use the verb Upload.   The process for uploading a file is more complex than a simple URL/CGI submission. Instead of an http GET request, a "POST" request is used. Programming with Upload commands requires using special tecniques. See the upload examples below in the Curl and Perl sections. Also see the documentation for the Upload Firmware and Upload ConfigFile commands.

Responses to Commands.

All Programmatic Interface requests will trigger a response from the NetRS. The response usually is a plain-text ASCII document which is returned using the HTTP or HTTPS protocol. The response can take one of several forms:

Single Line Data Response

These are usually the result of "Show" commands, where the data returned is simple enough to fit on a single text line. For example, the response to command Show SerialNumber will be:
    SerialNumber sn=1234567890
Single-line responses can be arbitary text, but most often they are similar to a Set command format, showing an object name with parameters. For example, Show GpsSatControls returns:
    GpsSatControls enable=all disable=5,6 ignoreHealth=7
This string is exactly what must be appended to the Set verb, in order to restore the controls to the current state.

Multi-Line Data Response

Some responses are too lengthy to be returned in a single line of text. In this case, the response will be spread over multiple lines, with leading and trailing lines that frame the data. Both framing lines will start with '<' and end with '>'. Between the two characters will be some text that shows which command was used to trigger the response. For example, the Show Sessions command produces a response like this:
    <Show Sessions>
    Session name=CORSA enable=yes schedule=Continuous ....
    Session name=CORSB enable=no  schedule=Continuous
    ....
    Session name=CORSC enable=no schedule=Continuous ....
    <end of Show Sessions>
Schematically, the multi-line format takes the form:
    <Verb Object Parameters>
    ....
    Zero or more lines of ASCII data.
    ....
    <end of Verb Object Parameters>
The following should be noted:

Single Line Action Response - 'OK:'

All commands that use a verb besides Show attempt to initiate some action besides sending some existing data. The response to these commands indicates two things. First that the command succeeded, and second, what the results of the action were. Successful execution of the command is always shown by a response line that starts with OK:. This is followed, on the same line, by descriptive text indicating what action was taken.

Often the descriptive text takes the form of the response from a corresponding Show command that shows the new state of the object being controlled. For example, the command Set Pps enable=yes slope=positive returns

    OK: PPS enable=yes slope=positive
Some commands allow partial parameter lists, and then use defaults for the unspecified values. In these cases, the OK response will always show the complete list of settings that the affected object now has.

In other cases the descriptive text will show an appropriate description of what action was taken. For example:

    Delete IOport port=TcpPort5066
    OK: IOport TcpPort5066 is deleted.

    Reset GpsData
    OK: Clearing GpsData and restarting satellite tracking operations.

A Single Line Error Message

If the command parser has difficulty understanding or implementing a command, it will respond with a single line error message. The message will indicate why the command was rejected, or why it failed to execute. Here are some examples:
    Show
    ERROR: Invalid command 'Show'

    Shw System
    ERROR: Invalid verb 'Shw'

    Show System
    ERROR: Invalid verb/object combination 'Show System'

    Show SystemName
    SystemName name=Boulder7

    Enable Session name=testing
    ERROR: No Session 'testing' exists.

    Set AutoDelete enable=maybe
    ERROR: Invalid parameter enable=maybe

Binary File Response

Some commands return a non-ASCII response, in the form of an 'application/octet-stream' binary document. This is only used by the Download LoggedFile command. Receiving such a document may require special processing. For details, see the description for Download LoggedFile.

How to Use Programmatic Commands

In order to use the Programmatic Interface commands, the following must be available:

Programmatic commands have to be encoded as URL or CGI requests. This requires assembling several pieces of information.

The URL will be of the form:
    http://SystemName/prog/Verb?Object&param=value&param=value...

If any parameter values have special characters in them, like spaces or ampersands, these must be encoded using %hex formatting. For example, each space must be replaced with the sequence %20.

Once the URL is determined, the CGI transmission tool is used to send a "GET" request, containing the encoded command, to the target NetRS. The NetRS will respond by sending back a document in one of the five response types. Almost all responses are simple ASCII text which can then be displayed or parsed as the application requires.

One exception to the previous paragraph is the case of uploading a file to the NetRS. Upload commands use a "POST" request instead of a "GET" request. See Uploading Files for details.


Using Curl

A simple method of testing out Programmatic commands is to manually feed the URLs into the curl, a commonly available command line tool. Here is a sample Unix session that shows the usage.

 $ curl 'http://NetRS27/prog/show?systemName'
 SystemName name=TomsProto

 $ curl 'http://NetRS27/prog/show?gpstime'
 GpsTime gpsweek=1244 weekseconds=437597

 $ curl 'http://NetRS27/prog/badcommand?abc'
 ERROR: invalid verb 'badcommand'

 $ curl 'http://NetRS27/prog/set?elevationmask&mask=10'
 OK: ElevationMask mask=10

 $ curl 'http://NetRS27/prog/show?position'
 <Show Position>
 GpsWeek     1244
 WeekSeconds 498154.0
 Latitude    37.3891271874 deg
 Longitude   -122.0368443968 deg
 Altitude    -4.898 meters
 ....
 <end of Show Position>

Multiple curl commands could be assembled into a shell script to implement a basic form of remote control. If the NetRS has Sysadmin Security enabled, an account name and password will be required in order to use the Programmatic Interface commands. Curl accepts these in a '-u' command line argument. For example:

  $ curl -u sysadmin:adminpw 'http://NetRS27/prog/show?systemname'
  SystemName name=TomsProto
That submits the request, using account name 'sysadmin' and password 'adminpw'.

Binary file downloads with Curl simply require directing the output to a file.

  $ name=NetRS27200311122315d.T00
  $ curl "http://NetRS27/prog/download?LoggedFile&path=/200311/$name" > $name
  $
Note that there is no command line response in this case. If an error occured (for instance, if the file didn't exist in the NetRS) the messages would end up in the file.

File upload commands require curl to format a POST request, with a binary attachment. Curl implements this with a -F option:

  $ f=/tmp/myConfigFile
  $ curl 'http://fbtc/prog/upload?configFile' -F configFile=@$f
  OK:  ConfigFile name=myConfigFile was successfully uploaded.

  $ f=/tmp/NetRS-1.0-2.ppc.rpm
  $ curl 'http://fbtc/prog/upload?firmwareFile' -F firmwareFile=@$f
  OK:  FirmwareFile NetRS-1.0-2.ppc.rpm was successfully uploaded.
Note that in those two examples, the name of the file when it reached the target NetRS was derived from the name of the original file from the sending computer. The name could be changed on the target system with:
  $ f=/tmp/myConfigFile
  $ curl 'http://fbtc/prog/upload?configFile&name=newConfig' -F configFile=@$f
  OK:  ConfigFile name=newConfig was successfully uploaded.

Using Perl

Perl is a powerful scripting language. The language comes with numerous library packages which allow it to be used to automate many complex tasks. It is also available on most operating systems, which makes it ideal for cross-platform applications.

Perl can easily be used to control a NetRS using the Programmatic Interface commands. A simple method uses LWP - a Library for WWW access in Perl. On Linux, use  man LWP  for overview documentation. This is a powerful and complex package, which cannot be documented here. Some sample programs show the basic techniques needed. The first shows how to encode basic URL requests:

  #!/usr/bin/perl -w
  use strict;
  use LWP::Simple;

  print get( "http://fbtc/prog/show?systemname" ) ;
  print get( "http://fbtc/prog/show?gpstime" ) ;
  print get( "http://fbtc/prog/badCommand?abc" ) ;
  print get( "http://fbtc/prog/set?elevationMask&mask=10" ) ;
  print get( "http://fbtc/prog/show?position" ) ;
Running that program produces the following output:
  SystemName name=NewName
  GpsTime gpsweek=1244 weekseconds=498371
  ERROR: Invalid verb 'badCommand'
  OK: ElevationMask mask=10
  <Show Position>
  GpsWeek     1244
  WeekSeconds 498373.2
  Latitude    37.3891241306 deg
  Longitude   -122.0368464236 deg
  Altitude    -4.078 meters
  ....
  <end of Show Position>

File Downloads are a bit more complex than just redirecting a get() request to a file, mainly due to the fact that the files can be arbitrarily large. A more complex syntax allows Perl to download and put the results directly into a file.

  #!/usr/bin/perl -w

  use LWP::UserAgent;

  my $f = 'NetRS200311110654c.T00' ;
  my $path = '/200311/11' ;

  my $ua = LWP::UserAgent->new ;

  my $req = HTTP::Request->new(GET=>
            "http://NetRS27/prog/download?loggedFile&path=$path/$f" ) ;

  my $res = $ua->request( $req, $f ) ;

When this is run, the logged file on the NetRS is copied to an identically named file in the local computer. Note that no text comes to standard output.

File uploads use a similar technique.

  #!/usr/bin/perl -w

  use strict;
  use HTTP::Request::Common qw(POST);
  use LWP::UserAgent;

  print "OKAY\n" ;
  my $fname = '/tmp/NetRS-1.0-1.ppc.rpm' ;
  my $ua = LWP::UserAgent->new;
  my $command = 'http://fbtc/prog/Upload?FirmwareFile' ;
  my $response = $ua->request( POST( $command , 
                                     Content_Type => 'form-data',
                                     Content => [ 'firmwareFile' 
                                                  => [ $fname ] 
                                                ]
                                   ) 
                             );
  print $response->content ;
Running that program produces
  OK: FirmwareFile NetRS-1.0-1.ppc.rpm was successfully uploaded.

Other techniques

It is quite feasible to use other methods to transmit the Programmatic Commands to the target system. For example one could write C or C++ programs to directly open socket connections to the NetRS and directly transmit the requests over those channels. This is moderately advanced programming and the details are beyond the scope of this document.

The Programmatic Test Page

A Brower-based Programmatic Interface Test page is avaiable as a development tool. This page shows samples of all commands, allows entry of arbitary variations, and shows the responses generated. Direct links are provided to the help pages on individual commands.

The Programmatic Interface page is divided into three sections.

To test a command, type it into the text entry field. Only the Verb, Object, and Parameters should be specified. Don't include the text 'http://systemname' in the command. For example, you could type in   Set?ElevationMask&mask=10.    As a convenience, the field separators, ? and & can be entered as spaces, so you could also type in    Set ElevationMask mask=10.    When the command is acceptable click on Send or hit the Enter key on your keyboard. The command will be displayed in the middle frame and transmitted to the command parser. The results of the command will appear in the bottom subframe. The same command can be resubmitted multiple times, or it can be modified and resubmitted.

Below the command entry box are a set of red buttons corresponding to various categories of Programmatic Commands. There are, for example commands dealing with Sessions, I/O ports, Satellites, etc. Clicking on any of these buttons will bring up a set of blue sample command buttons. The sample commands show all of the commands available in that category. Clicking on the button or its name will write the command into the text entry field and (optionally) submit the command to the command parser. Each sample command also shows a brief description. Just in front of the description is a   symbol that will open a help page window which provides documentation for that command.

By default, most sample commands are automatically submitted when you click on them. This behavior can be suppressed by de-selecting the AutoSubmit checkbox. When AutoSubmit is off, the sample commands are only entered into the text entry field. You can then modify the text before executing the command with the Send button. Certain sample commands that have serious consequences will NEVER be autosubmitted. These are all marked on the sample command link with a leading * in the button label.

You can use the samples to explore the programmatic commands, their variations, and their responses. Keep in mind that the commands that you test are live and truly are modifying the configuration of your system. If you issue a Reset System command, your system is going to reboot! If you execute Delete?LoggedFiles with a valid path to files, then the files are going to go away!

Commands that upload files to the NetRS cannot be exercised through this interface.   Upload ConfigFile and   Upload FirmwareFile will both produce the error message:

  ERROR: no document attached to upload.
The command Download LoggedFile will allow you to download logged data files, through your browser. You should be prompted for a place to put the file, just you would be if you downloaded the file using the normal GUI page.

List of Programmatic Commands

Below is a complete list of the Action-Object commands accepted by the programmatic interface. Each related group of commands has a separate page, accessible by clicking the group title. Clicking an individual command takes you to the specific information on that command.

Status Commands

All of these commands (except one) display some information from the NetRS. The information can be a static item, like a serialnumber, or something dynamic like the current time or receiver position.


Satellite Commands

These commands are associated with GPS or WAAS/EGNOS satellites.


Session Commands

These commands are associated with setup and control of data logging sessions.


Data File Commands

These commands are associated with setup and control of data logging sessions.


Configuration Commands

These commands control or show the state of a number of system functions.


Config File Commands

These commands implement an interface to Configuration Files, which can be used to save and restore the state of system controls.


Input/Output Commands

These commands control how Serial, TCP and UDP ports are configured.


Internet Commands

These commands control several Internet related functions.


Security Commands

These commands control access to controls through the Web server or the Programmatic Interface.


Firmware Commands

These commands are associated with updating the firmware programs in the NetRS.