David Janes' Code Weblog

November 14, 2009

Twitter uses WOEIDs – why you should care

ideas,maps · David Janes · 5:43 am ·

Because I don’t think this announcement got sufficient attention: Twitter is using Yahoo’s WOEIDs to identify locations in its Geolocation API.

WOEIDs are:

  • a numeric identifier for geographical places or regions, from a “place of interest” all the way up to continent
  • uniquely defined, for each place
  • hierarchically nested
  • related to each other by natural concepts, such as neighbor-of, sibling-of, etc.
  • managed, by Yahoo
  • a language independent way of talking about place
  • not necessarily tied to political boundaries, i.e. there’s a WOEID for the Bay Area

This announcement is important because:

  • it orders tweets into “containers” that can be used to find those tweets easily. In particular, there’s now codes that a machine can easily work with to find human concepts. Previous attempts at identification depended upon things such user entered hash tags for things such as a nearby airport, postal code, zip code, etc. and other concepts that don’t necessarily reflect what’s really going on
  • it encourages others to start building on WOEIDs, a non-Google way of identifying places

Read more:

Note that I have some concerns about how well WOEIDs will work when we start wanting them being dynamically/crowd defined down to the business level (e.g. sort of like what foursquare is doing; I don’t think they use WOEIDs though.)

June 27, 2009

How iPhone OS 3.0 delivers location services

iphone,maps · David Janes · 6:38 am ·

Alec Saunders has a seriously cool article on his blog about how your iPhone knows where it is. I got to experience how well this works yesterday when we used TimmyMe in the McCowan Road/Ellesmere Avenue neighborhood to grab a coffee pre-meeting:

Many folks assume that iPhone’s location services are GPS based. In fact, they’re not. [...]

Skyhook’s innovation is to augment that with WiFi hotspot locations. Why WiFi? It can be wonderfully accurate, to within 30 to 60 feet, or 10 to 20m which is roughly the same as GPS. WiFi can establish a fix within less than a second. And, WiFi is not vulnerable to overhead obstructions, so it can be used indoors.

Skyhook’s technology relies on a database of WiFi access points in over 2,000 cities (and growing). Much like Google, they use vehicles to drive cities, and using a laptop in the vehicle, they chart the location of WiFi access points. In addition, on iPhone as users use the mapping application, new locations and newly discovered WiFi access points are automatically added to the database. It’s a kind of automated crowdsourcing, based on usage.

My emphasis on that last sentence. Very cool.

December 23, 2008

Pipe Cleaner (II)

demo,maps,pipe cleaner · David Janes · 6:37 am ·

Here’s the latest evolution of Pipe Cleaner, mainly recorded here for historical interest. The big change is that there isn’t a separate outside template – everything is in the one index.jd file. The new directive is template, which can read and execute an outside module or actually produce the final output (as we see in the very last directive). I have not put this up as an independent demo.

#
#	Import the Python fire module
#	- used in: map from:"fire.GetGeocodedIncidents" to:"incidents"
#
import module:"fire";

#
#	Header for Google Maps popup
#	- used in: map from:"fire.GetGeocodedIncidents" to:"incidents"
#
#
set to:"fitem.head.map" value:"""
<h3>
<a href="#{{ IncidentNumber }}">{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}</a>
</h3>
""";

#
#	Header for the sidebar
#	- used in: map from:"fire.GetGeocodedIncidents" to:"incidents"
#
set to:"fitem.head.sb" value:"""
<h3>
{% if latitude and longitude %}
<a href="javascript:js_maps.map.panTo(new GLatLng({{ latitude }}, {{ longitude }}))">*</a>
{% endif %}
<a href="#{{ IncidentNumber }}">{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}</a>
</h3>
""";

#
#	Body for the Google Maps pop and the sidebar
#	- used in: map from:"fire.GetGeocodedIncidents" to:"incidents"
#
set to:"fitem.body" value:"""
<p>
Alarm Level: {{ AlarmLevel }}
<br />
Incident Type: {{ IncidentType }}
<br />
City: {{ City }}
<br />
Street: {{ Street }} ({{ CrossStreet }})
<br />
Units: {{ Units }}
</p>
""";

#
#	Convert all the incidents from the fire module
#	to the path 'incidents' using the mapping rules defined above
#
#	- incidents are used in "gmaps.js" and "gmaps.html"
#
map from:"fire.GetGeocodedIncidents" to:"incidents" map:{
	"latitude" : "{{ latitude }}",
	"longitude" : "{{ longitude }}",
	"title" : "{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}",
	"uri" : "{{ HOME_URI }}#{{ IncidentNumber }}",
	"body" : "{{ *fitem.head.map|safe }}{{ *fitem.body|safe }}",
	"body_sb" : "{{ *fitem.head.sb|safe }}{{ *fitem.body|safe }}",
	"IncidentNumber" : "{{ IncidentNumber }}"
};

#
#	Load the 'gmaps' templates (for arbitrary geo-mapping),
#	using the 'incidents' for its items and the specified meta.
#
#	- used in in "gmaps.js" and "gmaps.html"
#
set to:"map_meta" value_render:true value:{
	"id" : "maps",
	"latitude" : 43.67,
	"longitude" : -79.38,
	"uzoom" : -13,
	"gzoom" : 13,
	"api_key" : "{{ cfg.gmaps.api_key|otherwise:'ABQIAAA...pIxzZQ' }}",
	"html" : {
		"width" : "1024px",
		"height" : "800px"
	}
};

#
#	Produce GMaps
#
template script:"gmaps" items:"incidents" meta:"map_meta";

#
#	Produce the final output
#
template value:"""
<html>
<head>
    <link rel="stylesheet" type="text/css" href="css.css" />
	{{ gmaps.js|safe }}
</head>
<body>
<div id="content_wrapper">
	<div id="map_wrapper">
		{{ gmaps.html|safe }}
	</div>
	<div id="text_wrapper">
{% for incident in incidents %}
	<div id="{{ incident.IncidentNumber }}">
		{{ incident.body_sb|safe }}
	</div>
{% endfor %}
</div>
</body>
</html>
""";

The gmaps.jd (imported in the second last directive) looks like as follows (there will not be a test). It’s designed to be a universal “show a map and plot points on in it” inclusion. I’ve added a few line breaks so the PRE box doesn’t break.

#
#
#
template to:"html" value:"""
<div id="id_{{ meta.id|jslug }}"
style="{% if meta.html.width %}width: {{ meta.html.width }};{% endif %}
{% if meta.html.height %} height: {{ meta.html.height }};{% endif %}
{% if meta.html.style %} style: {{ meta.html.style }};{% endif %}"
{% if meta.html.class %} class="{{ meta.html.class }}"{% endif %}
></div>
<script type="text/javascript">
js_{{ meta.id|jslug }}.onload();
</script>
""";

#
#
#
template to:"js" value:"""
<script
 type="text/javascript"
 src="http://maps.google.com/maps?file=api&v=2&key={{meta.api_key}}">
</script>
<script type="text/javascript">
js_{{ meta.id|jslug }} = {
 onload : function() {
  js_{{ meta.id|jslug }}.map = new GMap2(document.getElementById("id_{{ meta.id|jslug }}"));
  m = js_{{ meta.id|jslug }}.map;
  m.setCenter(new GLatLng({{ meta.latitude }}, {{ meta.longitude }}), {{ meta.gzoom }});

  // {{ items|length }} items follow
{% for itemd in items %}
{% if itemd.latitude and itemd.longitude %}

  // {{ itemd.title }}
  var ll = new GLatLng({{ itemd.latitude }}, {{ itemd.longitude }});
  var marker = js_{{ meta.id|jslug }}.make_marker(m, ll, "{{ itemd.body|safe|escapejs }}");
  m.addOverlay(marker);
{% else %}
  // an item is missing latitude or longitude
{% endif %}
{% endfor %}
 },

 make_marker : function(m, ll, html) {
  var marker = new GMarker(ll);
  GEvent.addListener(marker, "click", function() {
   m.openInfoWindowHtml(ll, html);
  });

  return marker;
 },

 end : 0
}
</script>
""";

December 18, 2008

Pipe Cleaner

demo,djolt,dqt,html / javascript,ideas,jd,maps,pipe cleaner,pybm,work · David Janes · 6:38 pm ·

I’ve been working (in my decreasing available spare time) on a project to pull together into a project called “Pipe Cleaner” all the various concepts I’ve been mentioning on this blog: Web Object Records (WORK) for API Access and object manipulation, Djolt for generating text from templates, Data/Query/Transform/Template (DQT) for transforming data and JD for scripting these elements together. The pieces came together this morning enough to put a demo together and here it is – the Toronto Fires Pt II Demo.

How, you may ask, does this differ from the original Toronto Fires Demo? The answer is how it is put together, which we describe here.

Index.dj

This is the Djolt template that generates the output. The data fed to this template is generate by the JD script, described in the next section.

<html>
<head>
    <link rel="stylesheet" type="text/css" href="css.css" />
    {{ gmaps.js|safe }}
</head>
<body>
<div id="content_wrapper">
    <div id="map_wrapper">
        {{ gmaps.html|safe }}
    </div>
    <div id="text_wrapper">
{% for incident in incidents %}
    <div id="{{ incident.IncidentNumber }}">
        {{ incident.body_sb|safe }}
    </div>
{% endfor %}
</div>
</body>
</html>

Quite simple … as you can see, most of the data is being pulled in from elsewhere. The elsewhere is provided by the script described in the next section.

Index.jd

This is the script that pull all the pieces together. Note that I’m not 100% happy with the way the data is imported, I would like the geocoding to become part of this data flow too. In the next release perhaps.

First we pull in the “fire” module that we wrote in the previous Map examples. This is doing exactly what you think: importing a Python module. We may have to increase the security or restrict this to working with an API for general purpose use.

import module:"fire";

Next we define two headers – one that is going to appear in the Google Maps popup, the next that is going to appear in the sidebar. They need to be different as they refer to themselves. Note that the sidebar header “breaks” the encapsulation of Google Maps – this seems to be unavoidable. The to:"fitem.head.map" and to:"fitem.head.sb" are manipulating a WORK dictionary to store values.

Note also here that we’ve extended JD to accept Python multiline strings – this was unavoidable if JD was to be useful to me.

set to:"fitem.head.map" value:"""
<h3>
<a href="#{{ IncidentNumber }}">{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}
</h3>
""";

set to:"fitem.head.sb" value:"""
<h3>
{% if latitude and longitude %}
<a href="javascript:js_maps.map.panTo(new GLatLng({{ latitude }}, {{ longitude }}))">*
{% endif %}
<a href="#{{ IncidentNumber }}">{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}
</h3>
""";

The next block defines the text of the body used to describe a fire incident. It follows much the same pattern as the previous block.

set to:"fitem.body" value:"""
<p>
Alarm Level: {{ AlarmLevel }}
<br />
Incident Type: {{ IncidentType }}
<br />
City: {{ City }}
<br />
Street: {{ Street }} ({{ CrossStreet }})
<br />
Units: {{ Units }}
</p>
""";

This is a map: it is translating the values in fire.GetGeocodeIncidents into a new format and storing that in incidents. The format that we were are storing it in is understood by the Google Maps generating module.

We may rename this translate, as the word map is somewhat overloaded.

map from:"fire.GetGeocodedIncidents" to:"incidents" map:{
    "latitude" : "{{ latitude }}",
    "longitude" : "{{ longitude }}",
    "title" : "{{ AlarmLevel}}: {{ IncidentType }} on {{ RawStreet }}",
    "uri" : "{{ HOME_URI }}#{{ IncidentNumber }}",
    "body" : "{{ *fitem.head.map|safe }}{{ *fitem.body|safe }}",
    "body_sb" : "{{ *fitem.head.sb|safe }}{{ *fitem.body|safe }}",
    "IncidentNumber" : "{{ IncidentNumber }}"
};

Next we set up the “meta” (see WORK meta description if you’re not following along) for the maps. The render_value:true declaration makes PC interpret the templates in strings). We then call our Google Maps generating code (which are actually more Pipe Cleaners) and that gets fed to the Djolt template we first showed you. Clear? Maybe not, we’ll have more examples coming…

set to:"map_meta" render_value:true value:{
    "id" : "maps",
    "latitude" : 43.67,
    "longitude" : -79.38,
    "uzoom" : -13,
    "gzoom" : 13,
    "api_key" : "{{ cfg.gmaps.api_key|otherwise:'...mykey...' }}",
    "html" : {
        "width" : "1024px",
        "height" : "800px"
    }
};

load template:"gmaps.js" items:"incidents" meta:"map_meta";
load template:"gmaps.html" items:"incidents" meta:"map_meta";

December 7, 2008

Once more with the maps

maps · David Janes · 7:48 am ·

Michal Migurski added a very informative comment about tiling levels here, with a pointer to way more detailed information here.

November 22, 2008

Toronto Fires

demo,djolt,ideas,maps,work · David Janes · 4:19 pm ·

Here’s a little mashup I’ve been putting together for the last few days: Toronto Fires.

It’s taking the data listed here on the City of Toronto’s Fire Services “Active Accidents”, scraping it (by pretending HTML is XHTML and treating it as WORK objects), geocoding it (using our WORK Google API) and mapping it (using this information).

This is very much a work in progress, but here’s a few more things that are involved:

  • we read body.table.tr.td[1].table.tr[1].td.table.tr as a list to get the rows in the table
  • we map those rows into the Geocoder use a new magic technology we’ll be explaining in the next few days: DjoltDjango-like templates
  • the output program is just one big Djolt template

I’m not quite satisfied with how the current page is constructed: I want the final result to be much more simple.

November 14, 2008

How to make your blog readable on an iPhone

html / javascript,ideas,maps,mobile,tips · David Janes · 10:47 am ·

Here’s the following changes I made to this blog to make it readable on an iPhone

Add iPhone directives to header

  • the META tag informas the iPhone about how wide we want the page to look – i.e. the width of the iPhone
  • the LINK tag loads our iPhone specific CSS (tip from here)
  • this should be after your normal CSS LINK (or whatever) directive
<meta name="viewport" content="width=320" />
<link rel="stylesheet" type="text/css"
  media="only screen and (max-device-width: 480px)"
  href="/blog/wp-content/themes/davidjanes/iphone.css" />

Define iPhone specific CSS

  • this will obviously vary depending on what your current CSS does – that still gets loaded
  • most of this was pragmatically discovered
  • I hide several sections which I don’t think the user wants to see on an iPhone; I’ll probably play with this more
  • the PRE directive doesn’t use line breaking, so we just clip the examples, after making sure the font is small enough to get enough on a single line
body {
    padding: 5px;
    width: 480px;
}
div#content {
    float: none;
}
div#menu {
    display: none;
}
p.credit {
    display: none;
}
pre {
    overflow: hidden;
    font-size: 10px !important;
}
h1, h1 * {
    font-size: 36px;
}

What still needs to be done

  • we shouldn’t serve sections that users are not going to see – it’s a waste of bandwidth
  • we shouldn’t serve more than 10 articles
  • I’ll have to figure out how to do mobile browser detection on WordPress

How to enable/disable Mouse Wheel actions on your map

html / javascript,maps,tips · David Janes · 8:44 am ·

All the major map APIs have the ability to zoom in and out if your pointer is over the map and you scroll the mouse wheel. Being able to disable this function if you’re working in a small popup form window is very important!

Google Maps

By default, this feature is disabled. To enable:

map.enableScrollWheelZoom();

To disable (again):

map.disableScrollWheelZoom();

Source

Yahoo Maps

By default, this feature is enabled. To disable:

map.disableKeyControls()

There doesn’t appear to be a way to re-enable afterward.

Source

Microsoft Virtual Earth

By default, this feature is enabled. To disable you have to capture the event:

trap = function() { return true; }
map.AttachEvent("onmousewheel", trap);

To re-enable, you have to detach the exact same event (hence the trap function)

map.DetachEvent("onmousewheel", trap);

Source

November 9, 2008

Adding MapField to inputEx

demo,genx,html / javascript,ideas,inputex,maps · David Janes · 7:20 am ·

InputEx is a neat Javascript library by Eric Abouaf and other contributors to dynamically create forms. It’s best shown by example, so:

I’m working on a project called GenX that allows people to dynamically lookup, edit and create data in editors (such as in WordPress). One of the features I want is the ability to add maps so after a week of research, I’ve created a MapField for inputEx:

Notes

  • this is just a preliminary version; there’s probably some code style and idiom issues that need to be worked out
  • I’m going to ask the InputEx group to pick this up; I reserve no rights
  • it’s not fully integrated into the Builder example yet
  • it works with Google Maps, Yahoo Maps and Microsoft’s Virtual Earth. You will need a key for your domain for Google Maps, see the example above about how to insert your key
  • it introduces a concept of a Globals for MapField, as there are certain settings (i.e. your mapping keys) that really belong at “class” level rather than instantiated object level
  • Mapping APIs are dynamically loaded
  • Zoom levels are returned in native format and “universal format
  • I’ve done all my testing of FF3, so there may be browser issues I need to look at

Code changes

  • the only files needed are
    • examples/map_field.html
    • js/fields/MapField.js

Where this is going

  • I’ve spent the last week working on maps; I need to get back to using this code rather than improving and documenting it
  • I want to add a geocoder
  • I want the geocoder to be able to be sensitive to other form fields, so for example if you’re entering an address it can actually use those as an input

One final mod to inputEx

I’ve modified the builder to make the pencil icon more obviously become the close button when you’ve opened a subpanel. You can see the demo here. Here’s the change set:

In images:

curl --location http://www.famfamfam.com/lab/icons/silk/icons/pencil_delete.png \
 > pencil_delete.png

In inputex/inputExBuilder/css/inputExBuilder.css add:

div.inputEx-TypeField-EditButton.opened {
   background-image: url(../../images/pencil_delete.png);
}

In inputex/js/fields/TypeField.js replace onTogglePropertiesPanel with:

   onTogglePropertiesPanel: function() {
      if (this.propertyPanel.style.display == 'none') {
         this.propertyPanel.style.display = '';
         YAHOO.util.Dom.addClass(this.button, "opened");
      } else {
         this.propertyPanel.style.display = 'none';
         YAHOO.util.Dom.removeClass(this.button, "opened");
      }
   },

November 8, 2008

Switching between mapping APIs and universal zoom levels

html / javascript,maps · David Janes · 3:03 pm ·

Did you know that Google Maps, Yahoo Maps and Virtual Earth all use the same map tile resolutions? That is, you can actually seamlessly switch between mapping systems and have everything line up exactly the same way. If you’d like to play with this, look at our map examples:

But it’s even easier to show this using a graphic (thank you to Alistair Morton of Peapod Studios for compositing): All Maps

Table of map zoom level equivalents

Google Maps and Virtual Earth the same zoom levels, excepting that Google has a level 0. Bigger numbers are closer to the ground, smaller numbers are up into space. Yahoo Maps has fewer levels to work with – it doesn’t let you drill down below the block level. Yahoo numbers its zoom levels the opposite of Google and Virtual Earth, small numbers being closer to the ground and so on and so forth.

To allow seamless switching of data between APIs, I propose here universal zoom levels, which can be used as-is on any of these mapping APIs. To avoid confusion of which type of zoom level one is using, we use the negative number range for universal zoom levels. As Google and Microsoft are using the same zoom levels, I’ve decided that the universal zoom levels should simply be the inverse of their zoom levels, bounded by (-1, -19) inclusive.

Google Maps Virtual Earth Yahoo Maps Universal
Ground 19 19 - -19
18 18 - -18
Block 17 17 1 -17
16 16 2 -16
15 15 3 -15
14 14 4 -14
City 13 13 5 -13
12 12 6 -12
11 11 7 -11
10 10 8 -10
9 9 9 -9
8 8 10 -8
State 7 7 11 -7
6 6 12 -6
5 5 13 -5
4 4 14 -4
3 3 15 -3
2 2 16 -2
Space 1 1 17 -1
0 - - -

Conversion Formulae

Here’s Javascript to convert between your favorite mapping API’s zoom levels and universal zoom levels. Note that all to_native functions accept positive numbers as-is.

zoom = {
  google : {
    to_univeral : function(native) {
      return  Math.min(-1, -native);
    },

    to_native : function(universal) {
      return  universal > 0 ? universal : -universal;
    }
  },

  virtual_earth : {
    to_univeral : function(native) {
      return  Math.min(-1, -native);
    },

    to_native : function(universal) {
      return  universal > 0 ? universal : -universal;
    }
  },

  yahoo : {
    to_univeral : function(native) {
      return  Math.min(-1, native - 18);
    },

    to_native : function(universal) {
      return  universal > 0 ? universal : Math.max(universal + 18, 1);
    }
  }
};

End Notes

A truly universal zoom level would just use altitude; dealing with a simple integer is easier and less complication prone though. MapQuest is not included in this post as its tiles do not line up with the other APIs.

Older Posts »

Powered by WordPress

Switch to our mobile site