OSM Server Side Script

Introduction - Use Cases - Interface Calls - Statements - Concepts - Input Formats - Output Formats

Table of Contents

Introduction
Use Cases
    To which country belongs this location?
    Download an entire city
    Why doesn't my city appear?
    Declare a new type of area
Interface Calls
    /api/interpreter
    /api/get_rule
    /api/add_rule
    /api/update_rule
Statements
    area-query
    bbox-query
    conflict
    coord-query
    detect-odd-nodes
    foreach
    id-query
    item
    make-area
    osm-script
    print
    query
    recurse
    report
    union
Concepts
    Data structures
    Queries and Rules
    Control Flow
    Variables
    Other Sets in Scripts
    Flow Forecast
    Limits and Quotas
    Database Layout
Input Formats
Output Formats
    Payload
    Status Remarks
    Hypertext Reports
    Static Error Feedback

Introduction

Use Cases

To which country belongs this location?
Download an entire city
Why doesn't my city appear?
Declare a new type of area

The following examples are some sample use cases to give you an idea what you can do with the system. Of course, there are many more useful use cases, and you are kindly invited to add other use cases in the wiki.

To which country belongs this location?

To determine in which areas a location specified by latitude and longitude lies, just change the latitude and longitude in the following query to appriate values: If your browser asks you what to do, just open the file with a text editor.

Of course, you can send this query also via any other way of submitting input to /api/interpreter. You will recieve a gzip-compressed XML-file in an adapted OSM format that contains the areas the location lies in.

The first command, <coord-query lat="51.25" lon="7.15"/>, searches the database for every area that covers the location defined by the latitude and longitude values. Then it stores the result into the default set. Afterwards, the second command <print mode="body"/> prints the content of the default set. The mode="body" indicates there that it shall print the id of the area as well as its tags.

Download an entire city

This is in general a two-step process. For the first step, determine the id of the area to download, e.g. from the result of a coord-query like in the above example. Another way to determine the id is to take advantage of the way the areas obtain their id:

For example, the area representing the German city "Wuppertal" has id 3,600,062,478 because its pivot element is the relation 62,478 describing the borders of Wuppertal. In particular this means that area ids are as stable as the ids in the OSM database. If you don't know the id of the OSM entity in question, then can simply query it.

The second step is the query to actually download the area: This query will take 15-30 seconds before it gives a response depending on server load.

Again, you can send this query also via any other way of submitting input to /api/interpreter. You will recieve a gzip-compressed XML-file in an adapted OSM format that contains all nodes that lie in this area (including all the nodes on the boundary), all ways that are referred from such a node and all relations that are referred by such a way or such a node.

If you prefer to receive additionally all nodes any way refers to, you can change the query to: This query also will take some time depending on server load before it gives a response.

We briefly explain both queries and start with the first one: The statement <union> collects the output of the enclosed statements in line 2 to 6. The block starts with the query <area-query ref="3600062478"/>. This statement returns all the nodes that lie inside or on the border of the area with id given by the parameter ref and stores them in into the default set. In the third line, <recurse type="node-relation" into="rels"/> returns all relations that have as member a node from the input, in this case the default set, into the set "rels". As this happens within an union-block, the new content of the set "rels" is added to the result of the union-block as a whole. In the next line, <recurse type="node-way"/> puts into the default set the ways that refer to at least one node from the default set. In particular, the default set now contains only ways, the union-block has collected so far all nodes and ways and some relations. In the last line of the block, <recurse type="way-relation"/> puts into the default set all relations that refer to at least one way from the default set beforehand. The union-block again collects these relations. Now the union-block contains altogether the nodes within the area and the ways and relations that refer to such a node or way. As there is no output parameter given, this set is stored into the default set. The statement after the block, <print mode="body"/> then prints the entire content of the default set with all its details as specified by the value "body" of the parameter "mode".

The second query differs from the first one in two aspects: It exhibits the root element <osm-script timeout="180"> to change a global setting from its default: Without explicit timeout, the server forecasts the runtime of the query and refuses its execution if it could take too much ressources. With a manual timeout, the query will be accepted regardless of its predicted runtime, but it will be aborted if it exceeeds its timeout. The second difference is the second union-block: The first statement <item/> just makes the current default set part of the result of the union-block. The second statement <recurse type="way-node"/> replaces the default set with the set of all nodes that are referred by the ways in the default set beforehand. This result is collected by the union-block. Thus, the result now conatins all nodes that are referred by a way appearing in the set.

Why doesn't my city appear?

You may get into the situation that you don't find an area where you expect one. In particular, a coord-query like the first use case doesn't contain the data of an area although there exists for example a relation to create it. In this case, you can query the conflicts that prevented the respective area creation rule to create an area from this. The following query will try to find conflicts related to the relation of the German city "Wuppertal": You should replace "Wuppertal, Stadt" by a suitable value for you relation.
The server will return a human readable HTML document that lists the conflicts related to this relation; these are generated by user editable rules and should be self-explanatory. The result can be empty (and probably will be empty for Wuppertal because the respective area has successfully been created), but we simply can't leave an area in permanently broken state to show how the conflict feedback system works.

If you get an empty result from the first query, check whether there exists a relation at all:
This searches the data for all relations that match the given criteria, i.e. that have a tag with key "name" and value "Wuppertal".

If you already know the id of your relation, you can use instead the following to find conflicts:

We give a short overview of the used statement: the query <query type="relation"><has-kv k="name" v="Wuppertal, Stadt"/></query> searches for relations (this is specified by the parameter type) that have a tag with key "name" and value "Wuppertal, Stadt" (this condition is specified by the tag <has-kv k="name" v="Wuppertal, Stadt"/>. A full explanation of the quite versatile syntax of the query statement can be found in its documentation. The <id-query type="relation" ref="62478"/> by contrast finds the single relation with id 62478 as specified by the parameter ref. Both store their result into the default set because there is no other set specified. Then, the statement <report/> prints to the user all conflicts that are related to an item in its input, again the default set. It doesn't change any values in the query. Alternatively, the <print mode="body"/> simply prints the entire content of the default set.

Declare a new type of area

Not all relations that refer to ways give automatically rise to a relation. The server recognizes a certain relation (or any other designated set of ways) as an area if a rule matches this relation or set of ways. The amazing thing is that you (or anybody else) can add rules or change the existing ones. Now assume that you want to designate all relations with a tag with key "foo" and value "bar" to represent areas.

You can retrieve an existing rule to get a blueprint for our new rule. For example, the following query will return the rule that creates areas from administrative units:
This will give you a result like the following: 


<osm-script name="Area::Create_from_admin_level" version="1">

<query type="relation">
  <has-kv k="admin_level"/>
  <has-kv k="name"/>
</query>
<foreach into="rel">
  <union>
    <recurse type="relation-way" from="rel"/>
    <recurse type="way-node"/>
  </union>
  <make-area pivot="rel" into="odd"/>
  <detect-odd-nodes into="odd"/>
  <foreach from="odd" into="i">
    <union><item set="i"/><item set="rel"/></union>
    <conflict>In <item set="rel"/>, the <item set="i"/> is contained in an odd number of segments.</conflict>
  </foreach>
</foreach>

</osm-script>

We take the following modifications: In the first line, we replace the name of the rule by a new, descriptive name because we are adding a new rule. We remove the version tag because this will be chosen by the server. Then, we replace the two lines starting with has-kv by a single line that describes the designation for the relations in question. Then we post the new rule to add_rule:
You should recieve a message that the rule has successfully been added along with its new version number. If you recieve a the error that a rule with this name already exists, you have probably not changed the name in the above rule (and another guy already has left this example in the rule base).

To avoid cluttering the server, please complete this tutorial with removing your rule. Just call update_rule with the empty rule and the parameter replace set to the version number just recieved:
You should recieve a message that the rule has successfully been updated.

If you recieve the error that you should provide the last version-id, then call again get_rule on you rule:
The result contains the correct version number.

We conclude by explaining the statements in the rule "Area::Create_from_admin_level": The first line <osm-script name="Area::Create_from_admin_level" version="1"> is an explicit statement of the root element because a rule needs always a name and the name is provided as parameter of the root element. The version tag is added by the server to indicate which version of the rule it has returned - in our example the most recent version which is by incidence 1.

The first executable statement in the rule is the query-block <query type="relation"><has-kv k="admin_level"/><has-kv k="name"/></query>. This query-block searches for relations as specified by the parameter "type". It returns only relations that have a tag with key "admin_level" and a tag with key "name". For both tags, any values match the search criteria as there are no values specified in the has-kv tags. The results of the query are stored into the default set. A detailed description of the query syntax can be found in its documentation.

The <foreach into="rel"> statement regulates the control flow: The body is executed once for each element in the input set. As there is no input set specified, the input is taken from the default set. The output set "rel" contains during each loop solely the respective element from the input set.

The foreach-body starts with an union-block, containing the two statements <recurse type="relation-way" from="rel"/> and <recurse type="way-node"/>. The first one sets the default set to the set of all ways that are members of the relation in the input set "rel". The second sets the default set to the set of all nodes that are referred by ways from the default set. Because of the union-block wrapped around, the default set contains afterwards the ways that are members of the relation and their associated nodes.

The statement <make-area pivot="rel" into="odd"/> now creates the area: It takes input from two different sets. The first one not specified, thus being the default set, contains the ways and nodes that decribe the spatial extent of the area; it contains any point that is separated from the south pole by an odd number of way segments. The second input set is specified by the pivot parameter. It should contain exactly one element, and the set "rel", set by foreach, does so. The area's id and tags are derived from this element. The output of make-area is the created area if any. It is simultanenously written to the output set as well as the database of derived data. The set is in our case redirected to the set "odd" as we don't need the area anymore.

The <detect-odd-nodes into="odd"/> statement detects nodes that appear an odd times as starting or end points of a way. These nodes are exactly the nodes that prevent make-area from making an area. The detect-odd-nodes statement takes its input from the default set because there is no other set specified. The default set still contains the ways and nodes to produce an area from. The odd nodes are returned into the specified output set "odd".

The <foreach from="odd" into="i"> statement loops over these nodes, which are stored in the input set "odd". Each node is during one loop available as content of the set "i". In the loop, the line <union><item set="i"/><item set="rel"/></union> sets the default set to the node of the inner loop and the relation from the outer loop: The <item set="i"/> statement returns the content of the set "i" as output, hence the node of the current inner loop. The <item set="rel"/> returns the set "rel" that contains the relation of the current outer loop.

Finally, the line <conflict>In <item set="rel"/>, the <item set="i"/> is contained in an odd number of segments.</conflict> writes the conflict message into the database for the elements of the current inner and current outer loop. It obtains these elements from its input set, the default set. In the conflict message, the subtags <item set="rel"/> and <item set="i"/> are replaced by their contents. The conflict statement does not return anything. Then the rule continues with another inner loop or outer loop respectively.

Interface Calls

/api/interpreter
/api/get_rule
/api/add_rule
/api/update_rule

/api/interpreter

This is the API call to send a query, i.e. a request that returns the desired part of the OSM data. Some examples for queries are explained in "To which country belongs this location?", "Download an entire city" or "Why doesn't my city appear?". The exact possibilities of formatting input are explained in the chapter "Input Formats". The possible errors and the output format are explained in the chapter "Output Formats".

/api/get_rule

This API call returns the source code of an existing rule. An example how to use this API call can be found in "Declare a new type of area". To use this call you must send a request that consists only of the root element, with proper values for the attribute name and maybe a value for the attribute version. The returned data is a plain XML document without any compression.

/api/add_rule

This API call adds a new rule. An example how to use this API call can be found in "Declare a new type of area". To use this call you must send a request that containes the rule. This means in particular that you must specify the attribute name of the root element. The returned data is a plain XHTML document without any compression and tells you whether the rule has been added or why it hasn't been added.

/api/update_rule

This API call updates an existing rule. An example how to use this API call can be found in "Declare a new type of area". To use this call you must send a request that containes the new source code of the rule. This means in particular that you must specify the attributes name and replace of the root element. Name must match the name of the exsiting rule and replace must match the version of the existing rule. This is a protection against accidently overwriting a rule. You can always obtain the current version number of a rule by a call to /api/get_rule. The returned data is a plain XHTML document without any compression and tells you whether the rule has been updated or why it hasn't been added.

Statements

area-query
bbox-query
conflict
coord-query
detect-odd-nodes
foreach
id-query
item
make-area
osm-script
print
query
recurse
report
union

area-query

Mandatory parameters

Optional parameters

Description

The area-query returns all nodes that have coordinates within the provided area. A node is considered to be within an area if it is in the interior or on the boundary of an area. The nodes are stored in the set specified by "into". The area is selected by providing the id of the area to the argument "ref".

Properties

Examples

bbox-query

Mandatory parameters

Optional parameters

Description

The bbox-query returns all nodes that have coordinates within the provided bbox. A node is considered to be within the bbox if it is in the interior or on the boundary of the bbox. The nodes are stored in the set specified by "into".

Properties

Examples

conflict

Mandatory parameters

none

Optional parameters

Description

The conflict statement tags permanently (until the next rule application) one or more objects in the database as being invoked into a conflict. It is intended to be used in rules to mark objects that should be reviewed by a human in the context of the rule. The statement composes from the text and the item subelements within the element a message that should describe the conflict. Every object contained int the input set as specified by the parameter from is then tagged with this message. The messages for a given object can at any instant afterwards be query with the report statement.

Properties

Examples

coord-query

Mandatory parameters

Optional parameters

Description

This statement returns as the set specifed by "into" all areas that cover the location on earth specified by the coordinates lat and lon. An area is also returned if the given location lies on its boundary.

Properties

Examples

detect-odd-nodes

Mandatory parameters

none

Optional parameters

Description

This statements detects in a set of ways all nodes that appear odd times as a endpoint. For a couple of tasks like the derivation of an area from a set of ways, an essential precondition is that the ways fit together. Fitting together means in this context that there is no junction of an odd number of segments at any node, because it that case, it is not well-defined which points of that polygon are inner and which not. Inner nodes of a way are always adjacent to two segments within that way so that they can be ignored. The statement takes the ways from the set specified by the argument "from" and puts the detected nodes into the argument "into".

Properties

Examples

foreach

Mandatory parameters

none

Optional parameters

Description

The foreach statement executes the sequence of its inner elements once for each element in the set specified by the argument "from". In particular, the set specified by the argument "from" will be stored on an internal stack. Then, for each element in the set specified by the argument "from", the following things will happen: first, the set specified by the argument "into" will be set to contain solely the element the loop is dedicated to. Then, the statements in the loop are executed. Finally, the server continues with the next loop. The order of the indivdual loops is unspecified, only the order within a loop adheres to the order in the source code.

Properties

Examples

id-query

Mandatory parameters

Optional parameters

Description

This statement queries the database for the single object of the specified type and with the specified id. It returns as the set specified by the argument "into" the set consisting of this object or an empty set if the object doesnt exist.

Properties

Examples

item

Mandatory parameters

none

Optional parameters

Description

The item statement declares its argument as output. This is used within the union statement and the conflict statement to refer to an existing set.

Properties

Examples

make-area

Mandatory parameters

Optional parameters

Description

This statement creates an area from a multipolygon, writes this area into the database and returns this area as output. From a general mathematical point of view, one can define an area from its boundary as follows: you fix the boundary and a single point that is outside the area. Then a point is outside if and only if any curve to the outside point crosses an even number of segments. We use this definition and take as the outside point always the south pole and the set of ways in the input set (specified by the argument "from") as boundary. The make-area statement now requires that the input set contains all nodes that are referred by at least one way in the set. Furthermore, it requires that the ways fit together. This means that any node is referred by an even number of segments. Otherwise there would be a boundary with interior or the exterior on both sides. The area then consists of all points such that you have to cross an odd number of segments to reach the south pole. The area is tagged with all tags from the pivot element and obtains its id as follows:

The statement finally returns as the output set specified by "into" the set consisting of the newly created area. If a precondition fails, it writes nothing to the database, produces a status remark and returns the empty set.

Properties

Examples

osm-script

Mandatory parameters

none

Optional parameters

Description

This is the root element of all queries and rules. You may omit this element if you don't need to provide an explicit argument. Which arguments are useful depends on whether you write a query or a rule.

The parameters "timeout" and "element-limit" are used to assert that a query has a reasonable ressource consumption. It is only necessary if the flow forecast can't predict a modest ressource consumption directly. If the query exceeds its asserted limits, it will be aborted. The parameter "debug" controls how much meta information is included: if it is set to "quiet", no meta information will be added. If it is set to "errors", it displays only errors. If it is set to "verbose", it shows all the messages explained in Status Remarks. If it is set to "static", it won't run the query but display the flow forecast for this query.

For a rule, the parameter "name" provides the name of the rule. It is used to add a rule or to update an existing one, as well as to query a rule. The parameter "version" tells you which version a given rule has. As a safety measure, if you intent want to update a rule, you need to specify by the "replace" parameter the version of the rule to replace.

Properties

Examples

print

Mandatory parameters

none

Optional parameters

Description

The statement prints the content of its input set specified by "from" into the response document. The output contains only the objects' ids if mode is set to "ids_only". It contains the ids, the coordinates of nodes and the members of ways and relations if mode is set to "skeleton". It contains all the information about the objects including their tags if the mode is set to "body".

Properties

Examples

query

Mandatory parameters

Optional parameters

Description

The query statement finds all objects of the specified type that have all the provided tags. The search criteria is formulated with one or more subelements of the form <has-kv k="foo" v="bar"/> or <has-kv k="foo"/>. The strings "foo" and "bar" should be replaced by the respective tags to search for. The former element restricts the result to objects that have a tag with the given key and the given value, the latter element restricts the result to objects that have a tag with the given key and arbitrary value.

Properties

Examples

recurse

Mandatory parameters

Optional parameters

Description

The statement deduces from all nodes, ways or relations in the input set the referred nodes, ways or relations and returns them into the output set. Depending on the value of "type", the statement behaves as follows:

Properties

Examples

report

Mandatory parameters

none

Optional parameters

Description

The statement queries the database for conflicts recorded by the conflict statement. It prints all conflicts that are related to any object in the set specified by "from" into the response document. If there is no print statement in the query, the output is formatted as hypertext report.

Properties

Examples

union

Mandatory parameters

none

Optional parameters

Description

The union statement collects the output sets of its subelements and returns them as the set specified by "into". For each statement, the output set is the set specified by the parameter "into" or the empty set if the statement has no parameter "into". The only exception from this rule is the item statement. This statement designates its output set by its sole argument "set".

Properties

Examples

Concepts

Data structures
Queries and Rules
Control Flow
Variables
Other Sets in Scripts
Flow Forecast
Limits and Quotas
Database Layout

Data structures

OSM Server Side Script adheres to the data structure of the OSM main database as close as possible, but there are differences. This server is intended to respond to advanced queries for spatial data as fast as possible. It does neither provide a possibility to edit data nor the community support features of the OSM main database. The server mirrors the spatial elements from the OSM main database: nodes with their ids and tags, ways with their ids, members and tags and relations with their ids, members, roles and tags. The ids for the elements are the same as in the OSM main database.

Furthermore, the OSM Server Side Script server stores some useful derived data structures. At the moment, these are areas and conflicts.

Areas are stored with their id, their spatial extend and tags that describe them. The areas are derived by user mutable rules from the OSM data. Each area inherits its id and its tags from its pivot element. The tags are copied verbatim from the pivot element. The id is derived as follows:

Note that two different areas can't have the same pivot element.

Conflicts contain a message that a certain arrangement of elements does not meet the expectations set up by one or more rule. They consist of a message and they are attached to the nodes, ways and relations that are in conflict with the rule. They are generated by a conflict statement within a rule. They can be retrieved by the report statement.

The following data structures that appear in the current API of the OSM main database aren't stored here: we don't store timestamps, versions, changesets, any user information or historical data, i.e. deleted elements or old versions of updated elements.

Queries and Rules

The servers deals with two different pieces of OSM Server Side Script source code. The first are queries, the second are rules. Both are called scripts.

A query is sent to the OSM Server Side Script server by a client to retrieve data. Currently, the only useful method for this is via the interpreter API call. The query will then, if possible, be executed, and any output is sent back to the client.

Feel free to boldly experiment with queries. The entire system is designed to prevent runaway queries: your query will be analysed before it is processed. If it exceeds the time or space limit of 3 minutes or 10,000,000 element equivalents as explained below, you will recieve a flow forecast warning. You can override this safety stop by manually setting the timeout or element limit. If you don't get a response from the server within three minutes although you have not set a manual override, this is an error of the server. Please contact in this case the server admin via the email adress osm3s.admin AT lokove.de and attach your query.

A rule is processed by the OSM Server Side Script server to generate derived data. For example, the rule that generates the areas based on administrative boundaries is discussed in this section of the Use Cases chapter. Rules can be manipulated by every user via the following API calls: First, you can retrieve rules by /api/get_rule. Furthermore, you can add completely new rules by /api/add_rule. Finally, you can change existing rules by /api/update_rule. Note that the rules behave monotoneously. Thus, by adding an additional rule, you can let the server generate additional derived data. But you can never change existing derived data unless you edit an existing rule.

Existing rules are uniquely identified by their name and protected by their version number against accidental edits. You need to explicitly specify the existing rule version as version to replace (as explained at the update_rule API call) to change the rule. The rules are executed in ascending order of their version numbers, and of every rule only the most recent version (the version with biggest version number) is executed.

Control Flow

For queries, there are two stages: first, a query is analysed, then it is executed. Rules are executed without prior analysis. We will discuss in the following section the execution, the analysis will be discussed in the section Flow Forecast.

Basically, the scripts are executed line by line. Additionally, there is the foreach statement: this statement allows to loop over all the elements of a given set. Note that this is designed to terminate always: the behaviour of a foreach statement does only depend on the contents of the set when the control flow arrives at the loop. A change of the set during the loop does not affect the behaviour of the foreach statement. Thus, you can never produce a infinitely running program.

The server might do optimizations if they are semantically equivalent. This can change the control flow. For example, in a query statement the order of the has-kv substatements will be changed to apply the most restrictive condition first to speed up the query.

Variables

The handles on a particular piece of OSM data are the variables. They serve to transmit subsets of OSM data between different statements of a script. Each variable represents a set that can contain any number of nodes, ways, relations and/or areas. The content of a variable can change during every execution of a statement in the script, depending on the semantics of the respective statement.

Names for variables may be every string that is valid XML. The variable with name "_" has a special function and is called default set: Most statements take input, produce output or both. To specify which set of OSM data to take as input or where to save output into, you normally must provide the name of the respective set as argument to "from" (for input) or "into" (for output). But most scripts need few different sets or even just one simultaneously. Thus, if you use most of the time only one set, you can give it the name "_" and omit the entire "from"- or "into"-parameter. For example, the statements <print from="_"/> and <print/> are semantically equivalent.

If you use in a statement as output variable an already previously used variable, the previous contents of the variable are replaced at the end of the statement's execution with the set formed by the output of the statement.

Other Sets in Scripts

Beside the variables, there is one further place where sets of OSM data are stored, the stack. The stack is used by the statements foreach and union.

The foreach statement stores on the stack during the execution of its body a copy of its pivot set, i.e. the set it loops through. The variable specified as output of the foreach set contains during each loop a set with a single element of the pivot set; there is exactly one loop for each element of the pivot set.

The union statement stores during its execution the union of all the output sets of its substatements. This set is finally assigned to the variable specified by the "into"-parameter of the union statement.

Both sets are taken into account by the Flow Forecast when counting the element equivalents for the element limit.

Flow Forecast

The Flow Forecast should prevent the following two situations: first that the server crashes due to heavy load, second that a complex query runs for hours and block the server's ressources although the client has already gone away. The Flow Forecast is a static analysis that will be taken of every query before the query is executed. If the static analysis gives rise to doubts that the query can run with modest ressource comsuption, the query is not executed and the client gets back the results of the analysis. The ressource limits are explained in the next section. Although we try to make a good prediction, the forecast may overestimate the needed ressources for a query in some cases. Thus, the Flow Forecast can be bypassed by setting the timeout and/or element-limit parameter in the root statement of the query.

First, the control flow is predicted. In general, the server just follows the semantics of the control flow of the individual statements and collects the meta information for the other estimations. In a foreach statement, the computations in the individual loops must be independent from each other. If this is the case, the foreach statement is investigated only once. Otherwise, the analysis will result in an error.

The time usage estimates the expected runtime of the script. This is done as follows: for each statement, an execution time is predicted when the control flow prediction arrives at this point. This prediction is based on the type of the statement and the number of elements in the involved sets. In particular, this includes the output of a statement. For example, a query statement may vary in its runtime very much. But it has only output. Thus, the size of the output set is the only handle to predict the runtime of this statement. For the sake of simplicity, the spatial distribution of the data is not taken into account, only the estimated number of elements involved, discriminated between nodes, ways, relations and areas.

The space usage estimates the expected memory usage of the query. The relevant number for this is the number of elements of each type in the sets at the end of every statement. The amout of memory is derived from this by a weighted sum of the numbers of elements of different types: a node counts as a single element, a way counts as 10 elements and a relation counts as 20 elements. The estimation ends with an error if the weighted sum surpasses the memory threshold for a single query (currently 10,000,000 element equivalents). For statements which produce only output, this estimation depends only on the statement's arguments. For statements which derive their output from their input, the estimation for the size of the output sets depends on the size of the input sets.

Limits and Quotas

Each query is subject to limits and quotas. This protects the server from getting overcrowded by runaway queries. On the other hand, the worries about ressource consumption should bother the normal user as fewly as possible. Thus, the ressource management of the OSM Server Side Script knows two ways of ressource management. A standard query is assessed by the Flow Forecast. If this analysis accepts the query, no further restrictions apply to it.

A query with a manually set timeout will be stopped by the server after the specified amount of time. There is no limit to the amount of time you can sepcify. If you set a manual timeout, please ensure that your client is waiting long enough. Some browsers or other clients cancel a request after 60, 120 or 180 seconds unless you configure them not to do so. If the server aborts a query, you will find the error message "Your query timed out". You may get this message after a partial response or along with other messages.

Slightly more complicated are the space restrictions. In general, the amount of elements which are in the server's RAM simultaneously is limited by the amount of RAM the server has. Thus, the element limit of 10,000,000 element equivalents (these use roughly 500 MB of RAM) is a hard limit. You can assert a lower limit, but not a higher limit. The server aborts the query if it grows to a bigger size than its limit.

Database Layout

The database is designed to serve complex spatial queries. Note that the OSM database is an extraordinary big database. Thus, the speed of almost all queries is limited by the speed of the server's harddisks. We cope with this problem by keeping data that is spatially related as closely to each other on the disk as possible. This is combined with a rather fine grained data structure. We will give you details to clarify which queries are fast and which are rather slow.

The data is split up into the skeleton and the tags. The skeleton consists for nodes of their id and coordinates, for ways and relations of their ids and the ids of their members. For nodes and ways, the storage of the skeleton data is organised with respect to the quadtiles they belong to. As relations don't have a spatial position, they are organised by id.

The data for the tags is organised again spatially in a separate table. But we take into account that keys and values often repeat frequently (like highway=motorway). This favors queries for generic tags like highway=motorway as well as for specific tags like name=Elberfeld. A query for a tag with may values as for example name=* will be comparably slow because it reads and compares large portions of the tag data.

Another effect of the splitting in skeleton and tag data is that a print query is significatly faster if it does not refer to tags. This is the reason for the "skeleton" and "ids_only" option of the print statement.

Areas and conflicts are currently stored in MySQL MyISAM tables. These are anyway rather small amounts of data and should not constitute a bottleneck.

Input Formats

Basically, the input is a complete XML document with root element osm-script. There is no formal DTD but the allowed elements are exactly the statements as explained in the statement chapter of this manual. This includes for each statement its allowed subelements, called substatements in this context, and tags, called parameters in this context. The XML syntax is modified in two ways: To make input as brief as possible, you can omit some parts of the XML code. To make input as flexible as possible, you can encode the input in different manners. In the following sections, we explain both modifications in detail with the simple query <print mode="body"/>

You can always omit the file type declaration. For example, the input

and the input
are from the server's point of view equal.

If you don't need to specify any parameters in the root element, you can even omit the root element itself. So, even the input
is for the server the same input as the above examples.

We explain the different possible encodings. To make it possible to submit data via HTML forms, you can either send your data verbatim or prefix it by "data=" and escape all special characters to a sequence of a percent sign, followed by the value of the respective byte as hexdecimal number. Multi-byte characters should be expanded in a individual sequence for each byte. For example, the inputs 


<print mode="body"/>

and 


data=%3Cprint%20mode=%22body%22/%3E

and 


data=%3Cprint%20mode=%22body%22/%3E&this_garbage_will_be_ignored

and 


foo=bar&data=%3Cprint%20mode=%22body%22/%3E&this_garbage_will_be_ignored

are equal. The transformation from the former to the latter is normally done by your browser or client program automatically or even from the users point of view unintentionally. So if you get a encoding error, try prefixing your query by "data=". The server tries to interpret your input verbatim if the first non-whitespace character found is a verbatim "<", in all other cases it searches for a "data=" substring, takes and decodes the input from that position until the next verbatim "&".

We explain the different possible encodings. You can send your data by a HTTP GET request as well as by a HTTP POST request, with precedence for the GET payload if you send both simultaneously. To continue the above example, the URL 

  http://78.46.81.38/api/interpreter?data=%3Cprint%20mode=%22body%22/%3E

is equal to any of the above queries.

If you want to use the famous program wget to retrieve the data from the server, you can use any of the sample command lines 


wget -O - http://78.46.81.38/api/interpreter?data=%3Cprint%20mode=%22body%22/%3E | gunzip

or 


wget -O - --post-data="data=%3Cprint%20mode=%22body%22/%3E" http://78.46.81.38/api/interpreter | gunzip

or 


wget -O - --post-data="<print mode=\"body\"/>" http://78.46.81.38/api/interpreter | gunzip

Note that in the third variant the quotation marks are escaped in the usual shell style. The output is piped into gunzip to make it visible as uncompressed XML. A more appropriate way for a longer query would be to save your query into a file, e.g. foo.xml and then use the command 


wget -O - --post-file=foo.xml http://78.46.81.38/api/interpreter | gunzip

Output Formats

Payload
Status Remarks
Hypertext Reports
Static Error Feedback

Payload

If the query has been answered successfully, you will receive a gzip compressed XML document of MIME type "application/osm3s". It is, as specified by HTTP, announced with header "Content-Encoding: gzip". Thus, your client may uncompress the data before storing it.

The root-tag of the XML document is "osm-derived". Furthermore, each document contains an attribution note and, in the tag "meta", a timestamp and an information which rule was applied. The timestamp is the timestamp of the OSM data that has been used for the query.

In the remainder, you get the data your query produced. In general, these are nodes, way and relations in a similar syntax as the OSM API but without version, user data and timestamp. The areas have the same syntax for tags as nodes, but they only have an id. Most likely, this is data from a print statement.

Status Remarks

Basically, you can get three different types of runtime remarks, state remarks, runtime remarks or runtime errors.

The state remarks provide you with information about how much time a certain statement consumed and which output was produced by which statement. They are included if you set in the root statement the parameter debug to verbose. They are tagged as <remark type="state"> and included after the termination of each statement. They contain a timestamp at what time they have finished, the statement's line number and the contents of the stack at that moment.

The runtime remarks inform you about important events that appeared during your request but that don't mean an error. At the moment, only the add_rule and update_rule api calls use this kind of remark to inform you that your request has been successful.

The runtime errors inform you about problems that definitely made your query to fail. The most common would be a timeout or out of memory error. But also a lot of unrecoverable database errors will be treated that way. Or ways that refer to nonexisiting nodes. In most of these cases you can't do much yourself, but please write an email to osm3s.admin AT lokove.de.

Hypertext Reports

If your query contains only report statements as statements with output, the server answers your quey with a human readable HTML document. It contains messages that have been generated by conflict statements during the rule computation. Each is presented in a paragraph. If the message contains nodes, there will be direct links to the respective location on the OpenStreetMap.org map.

Static Error Feedback

The static error feedback will be displayed if the server can't make a feasible query out of your input. It consists of up to four sections, "Encoding Remarks", "Your input formatted for XML parsing", "Static Remarks" and "Flow Forecast". The later sections don't appear if in the earlier sections an error has been detected.

The section Encoding Remarks contains information how the server has extracted the input from the incoming data. Details about the possible formats are explained in the chapter Input formats. The server tells you whether it has found data by method GET (that means, it has found something that looks like data in the request URI) or uses data from PUT or has not found any data at all - this will be indicated as "only whitespace found". The next issue is the encoding: either your data starts with a "<", or it is assumed to be CGI encoded. Again, the server tells you in this section what it assumes your data to be. The third thing that's detected automatically is whether your query has an explicit root statement and DTD declaration. If no root statement or DTD declaration is found, the server adds one and tells you that it has done so.

The section Your input formatted for XML parsing consists of a pretty printed version of what the server has taken as input from the client. If this does not match your intended input, probably something has gone wrong during the input decoding. If your input is properly displayed, the problem appears in a later step of processing. If there exists a static remark for a line, it is displayed in green and a link with letter "R" in parentheses is appended. If there exists a static error related to a certain line, the line is displayed in red and a link with letter "E" in parentheses is appended.

The section Static Remarks contains for a feasible query only remarks, no errors. The remarks contain information that might be useful for further analysis and do not indicate any trouble with the script. The errors indicate that your script is syntactically incorrect. There are errors that are common to most statements, errors which are specific for some statements and errors that appear only with rules.

The common and general errors include:

There exist the following errors related to a specific statement:

You may get the following errors on the root element if you perform another interface call than /api/interpreter:

The only existing remark so far is:

The section Flow Forecast contains the Flow Forecast analysis. This analysis will be displayed as follows: Each statement is printed in the order of the control flow. After each statement, in a section Forecast, the estimated time elapsed at this point is displayed. Also for each existing set at this point, its name and the estimated number of nodes, ways, relations and areas contained in the set are printed. If you see the error message Time exceeds limit of 180 seconds., your query is estimated to exceed the default timeout of 180 seconds and you need to manually set a timeout. If you see the error message Number of elements exceeds limit of 10,000,000 elements., your query is estimated to exceed the amount of element equivalents limit. Again, you need to manually set a limit for the amount of element equivalents.