Name
|
Just van den Broecke
|
Organisation
|
Just Objects B.V.
|
Email
|
just@justobjects.nl
|
Paper Abstract (short)
|
stETL, Streaming ETL and pronounced "Stetl", is a lightweight, geospatial ETL-framework written in Python, that tightly integrates and combines existing transformation tools like GDAL/OGR, XSLT and PostGIS. stETL targets ETL-cases that involve complex XML and GML data, like INSPIRE Data Harmonization but any other transformation, even non-geospatial, can be applied as well. stETL applies declarative programming: a configuration file specifies an ETL chain of input/filter/output modules. Standard modules are available or can be user-defined. stETL is speed-optimized, by using native calls to for example GDAL/OGR, libxml and libxslt.
|
Paper Abstract (long)
|
Data conversion combined with model and coordinate transformation from a source to a target datastore (files, databases) is a recurring task in almost every geospatial project. This proces is often refered to as ETL (Extract Transform Load). Source and/or target geo-data formats are increasingly encoded as GML (Geography Markup Language), either as flat records, so called Simple Features, but more and more using domain-specific, object oriented OGC/ISO GML Application Schema's. GML Application Schema's are for example heavily used within the INSPIRE Data Harmonization effort in Europe. Many National Mapping and Cadastral Agencies (NMCAs) use GML-encoded datasets as their bulk format for download and exchange and via Web Feature Services (WFSs). As geospatial professionals we are often confronted with ETL-tasks involving (complex) GML or worse: "GML-lookalikes", which are often XML Schema's embedded with GML-namespaced elements. Luckily, in many cases, GDAL/OGR, the Swiss Army Knife for geo-data conversion, can do the job. If "ogr2ogr" sounds like gibberish to you, check out http://gdal.org ! But when complex, some say rich, GML Application Schema's are involved, data conversion can be a daunting task when GDAL/OGR alone is not sufficient. Firstly, often complex data model transformations have to be applied. In addition we may be confronted with the bulkiness of GML: Megabyte/Gigabyte-files, deeply nested elements where the nuggets, the actual attribute values, reside, trees of .zip files and possibly more nasty surprises once we have unboxed a GML-delivery. High resource consumption in memory and CPU and long processing hours, up to complete machine-lockup, can be the the side-effects of naive GML-processing. Within the FOSS4G world we can resort to higher level, GUI-based, ETL-tools such as GeoKettle, Humboldt tools and Talend GeoSpatial. These are very powerful tools by themselves, check them out as well. Some of us, like the author, like to stay closer to GDAL/OGR, to XSLT for model transforms, some command line tools and some Python scripting, but without having to write a complete, ad-hoc ETL-program each time. This is the space where stETL tries to fit in, so read on. So we already have great FOSS tools for XML/GML parsing, data-conversion and model-transformation like GDAL/OGR (ogr2ogr!), XSLT (Extensible Stylesheet Language Transformations, for transforming XML) and native XML-parsing libraries like libxml2. Each individual tool/library is extremely powerful and performant by itself. But we would like to combine of these tools. Take for example flat, national adres data in a PostGIS database that we need to transform to multiple INSPIRE Application Schema GML files. Each individual FOSS tool can handle part of the ETL: ogr2ogr for converting from PostGIS (including coordinate tranformation) into to simple feature GML, XSLT (xsltproc/libxslt) to transform the resulting flat GML to rich INSPIRE GML. But with millions of addresses we cannot simply use a single GML memory datastructure (DOM) or single intermediate GML-file. Add Python and a configuration convention to this equation and we have stETL: Streaming ETL. stETL is a lightweight, geospatial ETL (Extract Transform Load) framework written in Python. ETL-processing with stETL is driven from a configuration file. Within a stETL configuration file a chain of ETL-processing modules is declared through which the data flows ("streams"). A module may be an input, filter or output module. Modules have input and output data types declared such that only compatible modules can be connected. However, stETL does not define a grand internal data structure to which all data is mapped as many ETL-tools do. Data formats are kept close to the external tools that stETL uses. stETL comes with pre-defined modules for GML-parsing, XSLT processing, XSD Validation, PostGIS/OGR input and output, GML-splitting and many more. stETL calls on the above tools like OGR, libxslt and libxml2 via their native interfaces. stETL is even more speed-optimized as no intermediate file-storage is used and through other means such as native string buffers. For example large XML/GML-files can be split into manegeable documents and streamed into an XSLT filter module. stETL-modules are offcourse extensible and can be user-defined. Reusable ETL-configurations invoked through parameterized commandline scripts can be defined without programming. stETL evolved from and is used within the INSPIRE-FOSS project (http://inspire-foss.org). Here for example, Dutch national addresses (BAG) were transformed into INSPIRE Addresses GML (files and database). Special stETL integration modules are available to extract and publish data from/to a deegree WFS and deegree "Blobstore-database". The combination stETL/deegree is an ideal setup for INSPIRE deployments. Other Dutch national datasets like Top10NL and BGT (Dutch topo vector datasets) have been completely and successfully transformed. Work is in progress to use stETL as the basis for NLExtract (http://nlextract.nl), a project that provides ETL tools for Dutch open geo-datasets. stETL development is now (april 2013) in an initial phase and takes place in GitHub. The current version is workable but we hope to present a v1.0 at FOSS4G with more documentation and as a standard Python Package via PyPi. The main link is: http://stetl.org (now links to GitHub). To get started find some basic examples here: https://github.com/justb4/stetl/tree/master/examples/basics. This presentation will gently introduce the above "GML challenge" and explain the basics of stETL starting with some simple examples, building up to more complex cases like INSPIRE transformations and integration with the deegree WFS/WMS server (http://deegree.org).
|
Topic type
|
|
Target Type
|
Development: new developments in products.
|
|
Technical / Developer
|
Name
|
Manuel Haro M‡rquez
|
Organisation
|
National Open Source Laboratory, Zacatecas State Goverment
|
Email
|
manuel.haro@zacatecas.gob.mx
|
Paper Title
|
Technological Innovation with Open Source
|
I can give a practical demo
|
yes
|
Paper Abstract (short)
|
At the society and economy knowledge age where the technologies are the development platforms for the countries, is important to create strategies wich ensure the advantage of all that scientific and technological developments towards knowledge appropriation, impulse the creativity and innovation going to create collaborative communities wich may guarantee a technological sovereignty and open innovation. Today Open Source, with their freedoms and robust solutions, is the best alternative and is a technological tendency towards I+D+I2 paradigm (Investigation + Development + Innovation + Inversion) where is very important the integration of academic, industrial, governmental and social sectors at the projects and those commitment.
|
Paper Abstract (long)
|
Technological Innovation with Open Source Ing. Manuel Haro M‡rquez Science, Technology and Innovation Zacatecan Council National Open Source Laboratory email: mharo@cozcyt.gob.mx, manuel.haro@zacatecas.gob.mx Abstract At the society and economy knowledge age where the technologies are the development platforms for the countries, is important to create strategies wich ensure the advantage of all that scientific and technological developments towards knowledge appropriation, impulse the creativity and innovation going to create collaborative communities wich may guarantee a technological sovereignty and open innovation. Today Open Source, with their freedoms and robust solutions, is the best alternative and is a technological tendency towards I+D+I2 paradigm (Investigation + Development + Innovation + Inversion) where is very important the integration of academic, industrial, governmental and social sectors at the projects and those commitment. I. INTRODUCTION The FOSS and GPL licensed solutions offers key points like freedom to use those at any instance and everyone machine, the solutions are shared ready to install and the code source wich allows modify to improve the solution, after the modify the solution must to be shared ready to use and with the code source. Any FOSS solution offer several ways to develope some project as advantage of developed solutions wich like a FOSS project the source code can be modified and installed, in other way some times the project must to be developed fully like a new solution; another kind of project with advantage of FOSS is using the solutions availables but this kind of projects need two importants parallel activities like a strong capacitation program and supplemented with a great support team to attend any incidence around the project. II. NATIONAL OPEN SOURCE LABORATORY The Open Innovation (OI) paradigm has been the best strategie at countries like Brasil, France, Spain and Germany wich allows to impulse the entrepreneurship and sectorial project development over open source developments, at National Open Source Laboratory (NOSL) we are working to apply some of that projects and strategies in the way to propitiate the global benefit for universities, goverment dependencies, companies, organizations and citizens. The strong of OI is the integration of communities at universities, companies, citizens and goverment and with the capabilities of each sector create global projects. In this way the academical sector is very important because the young students, teachers and researchers working together have a great potential of creation, research, share and development knowledge; the companies focussed to information technologies and services must to change the bussines model to advantage the open source solutions; the integration of society is a key point because the citizens can contribute with proposals focussed to improve social services. The goverment sector must to take the leadership building strategies to support and strengthen any kind of inititative of innovation projects, this leadership is about create important investment funds wich may be possible the succesful development of all kind of innovation projects. Integrating the elements of OI can ensure projects over a collaboration culture, the exploitation and development of open source solutions provides great alternatives to any kind of technological and innovation project. The National Open Source Laboratory achivied the integration of companies, universities and citizens to create the projects development teams, as a goverment dependencie take the leadership with important investment in infraestructure, equipment, services and scholarships for students working at projects; the semester august to december 2012 were integrated 28 students form Zacatecas Institute of Technology who developed 10 technological projects, today we are working with 49 students and the better is about this students is coming from 8 universities at Zacatecas State who are developing 15 projects. Listed below the 2012 projects: 1. PBX Server with Elastix 2. Web Page www.conlibre.org 3. Web Page NOSL www.cozcyt.gob.mx/labsol 4. Collaborative network with Noosfero 5. Services using IPv6 6. Open source migration project 7. Innovation management projects 8. LAN Services server 9. LAN and WAN Security Server 10. Mobile apps for android The NOSL works creating innovation projects to resolve requirements in any sector, all the projects are developed using open source solutions and considering the General Public License as FOSS development; so all the projects must to be share with documentation, install packages, source code and are available at NOSL repositories; when some instance wants to use any project we contact those students who works the project trying to impulse them to start work creating new companies focussed to advtange FOSS solutions and innovating with a new bussines model. Listed below the january to june 2013 projects: 1. Global Goverment Planning Platform 2. Documental Management with Quipux 3. Network security server with Squid and Iptables 4. Robotics with NAO humanoid Robot 5. NOSL intelligent bulding with RFID 6. Multimedia Learning Objects to science 7. Augmented reality 8. COZCyT open source migration project 9. COZCyT Network redesign 10. Parallel computing 11. COZCyT Web Page develope 12. ZigZag Science divulgation park WebPage develope 13. B-learning NOSL platform Listed below wich are ready to starts: 1. Global GIS Platform 2. OSGeo Laboratories Network Web Page develope 3. New Linux distribution design 4. Technical support to Federal Goverment equipment project (4,000,000 computers with linux) The NOSL strengthens their strategies with strategic alliances: Intel Corporation Brasilian Public Software Project, Federal Goverment of Brasil Pixart Corporation, Argentina Forest House Foundation, Colombia Latinux International Corporation Minatitlan Institute of Technology Puebla Institute of Technology Zacateas Institute of Technology Fresnillo Institute of Technology Technological University of Zacatecas Guadalajara University Working hard to create a strong culture using and developing open source solutions, the NOSL have a complete program of capacitation with almost 40 different courses, the alliance to Latinux International brings the way to apply for certification in some themes about open source and information technologies. The NOSL have enough infrastructure wich allows works with a lot projects and almost any kind of project: 3 Classroom for 20 students 1 Super Computing Laboratory 23 Dell Workstations with Core I5 Intel, 8 Gbytes RAM and 500 Gbytes in hard disk 60 Intel Classmate computers 8 Network servers Assigned to NOSL are 4 professional engineers who works as leaders of projects, with 15 collaborators from universities and some goverment dependencies who works in capacitation programs, special projects and academical background for the projects.
|
Topic type
|
|
Target Type
|
Case Studies: Relate your experiences.
Business Cases: building the economic case.
Development: new developments in products.
Collaboration: data collection, data sharing, open standards.
Open Innovation
|
|
People new to open source geospatial
Manager
Technical / Developer
|
Name
|
Andrea Antonello
|
Organisation
|
HydroloGIS S.r.l.
|
Email
|
andrea.antonello@gmail.com
|
Paper Title
|
The Android platform, mobile for everyone at the end of a tough trip? + Geopaparazzi
|
I can give a practical demo
|
no
|
Paper Abstract (short)
|
This presentation wants to trace back the history of the mobile evolution in the last decade + talk about Geopaparazzi
|
Paper Abstract (long)
|
The last decade has shown an initially crawling, but then explosive growth of the availability of the mobile technologies for the average user. While not so much time ago quite some investment was required for the professionals interested in trying out mobile technologies, now everything needed lies in the hands of a 12 years old kid that uses it to listen to music and chat in facebook. This presentation wants to trace back the history of the mobile evolution in the last decade, as seen through the eyes of a group of technicians, of endusers, professionals and passionates of the free and open source technologies. The presentation will conclude describing the Geopaprazzi project, an Android application for digital field mapping, and some projects that were built on top of it.
|
Topic type
|
|
Target Type
|
Case Studies: Relate your experiences.
Collaboration: data collection, data sharing, open standards.
|
|
People new to open source geospatial
End User
Technical / Developer
|
Additional Presenters
|
Name
|
Silvia Franceschi
|
Organisation
|
HydroloGIS S.r.l.
|
Email
|
silvia.franceschi@gmail.com
|
Name
|
BJ Jang
|
Organisation
|
Gaia3D Inc.,
|
Email
|
bjjang@gaia3d.com
|
Paper Abstract (short)
|
Gaia3D Inc., will introduce not only the experience of implementing mobile traffic map service (iPhone App, Android App, and Mobile Web Client) showing traffic on roads, streets, and highways at NTIC(National Transport Information Center) using Squid Proxy Server, GeoServer, and SQL Server but also advanced architecture coming up in 2014
|
Paper Abstract (long)
|
MOLIT(Ministry of Land, Infrastructure, and Transport) has established NTIC(National Transport Information Center) for effective management of various kinds of transportations in South Korea and released several services that people can use. Gaia3D Inc., has involved in one part of mobile service which displays traffic status on roads, streets, and highways up on geographical map, making people easily check the status of traffc wherever theyÕre heading. Gaia3D Inc., will introduce not only the experience of implementing mobile traffic map service (iPhone App, Android App, and Mobile Web Client) showing traffic on roads, streets, and highways at NTIC using Squid Proxy Server, GeoServer, and SQL Server but also advanced architecture coming up in 2014. NTIC system collects all kinds of real time traffic data of all highways, routes, streets, and roads in South Korea and divides those collected traffic data into three colors in green, yellow, and red by speed. These colorized traffic data are mashed up with map data to service on mobile devices. Servers carry out tiling traffic map in every 5 minutes and clients receive and display those tiled data. This system aimed at tolerating peak times of two major holiday seasons in South Korea - Chuseok(Korean Thanksgiving day) and Seolnal(Lunatic New YearÕs day) when almost 15 million people per day travel at the peakest dat and about 8 million vehicels are poured out to roads, streets, and highways, so the system should be designed to safely handle over 100,000 concurrent connections. The whole system is consisted of two Cache Servers with Squid Proxy, six Map Server with GeoServer, and three Database Server with SQL Server. Real time traffic information and road lines are managed in SQL Server and provided to GeoServer. Traffic map tiles are produced in GeoServer and are passed to Cache Server. The client is designed to request tiles via interface of WMTS(Web Map Tile Service) protocol with Time Tag. The very initail architecture designed in 2012 somehow managed to endure traffic loads at peak times, but had some problems, which was quite disappointing and unexpected results. In order to improve the system, weÕve mainly focused on the enhancement of scalability. Also, weÕve newly redesigned the system into seperating tile producing servers and managing static contents using NGINX web server.
|
Dostları ilə paylaş: |
