4D Cities (August 2009 to August 2010)

Frank Dellaert

Georgia Tech PI: Frank Dellaert
Students: Grant Schindler, Matt Greenland, Vince Thiele, Brian Lawrence
Samsung Technical POC: Namwook Kang

Project Goals

The goal of this project is to develop a platform for the creation and viewing of 4D city models – time varying 3D models of cities interactively constructed from modern and historical photographs. This has involved the creation of a client-server architecture to enable a 4D server to perform the underlying 3D reconstruction computations and to deliver content to a 4D builder and a 4D viewer.

Project Details

Client/Server
The 4D server has two jobs: to deliver existing 4D models (including 3D geometry, time information, and images) to the client, as well as to manage point-correspondences between images and perform optimization of the 3D model in order to recover scene geometry and camera parameters. The server is written in the Objective Caml programming language (OCaml) and runs in a Linux environment.
The 4D client has a number of roles: it can act as a 4D builder, a 4D viewer, or a web –based method for a user to add images to a model. We have developed the client in Java, so that it can be run on multiple platforms (Windows, Mac, Linux) and also run as either a standalone application or embedded in a web browser.
The 4D Client now supports the following tasks: Creating new models from a pair of images, Adding images to a model, Adding 3D points to a model, Adding buildings to a model, Specifying image dates, Specifying building dates.
Accomplishments

Delivered client/server architecture
Developed new split-screen interface for specifying correspondences between images
Integrated 4D city viewer with client/server architecture
Developed novel technique for starting new 4D city models from a pair of images, based on four initial points that specify the X, Y, and Z directions of the global coordinate frame
Developed web-based interface for users to contribute images to 4D city models

Future Work

Google Earth integration
Mobile augmented reality
Geo-tagged user photo collections

screenshots

Figure 1 . The primary interface for building 4D models is a pair of active images as seen in the top portion of the above screenshot. Corresponding points are specified by dragging from a point in the left image to a point in the right image. This information is sent to the server to update and optimize the 4D city model. The scrolling list of image thumbnails on bottom is used to select the active images.

Model in 4 D viewer interface.

Figure 2. After model creation, as shown in the above figure, the resulting models can immediately be viewed in the 4D viewer interface to ensure that optimization has arrived at a correct result. Here the model can be explored in 3D by orbiting above the scene or by viewing the scene from the viewpoint of each image.

Research Artifacts

4D City Builder Software: a client/server platform for the creation and viewing of time-varying 3D city models from historical and modern photographs. (2010)