source: other-projects/tipple-android/osmosis/changes.txt@ 29201

Last change on this file since 29201 was 26899, checked in by davidb, 11 years ago

Tipple reborn after Chris's Summer of Code 2013

File size: 21.9 KB
Line 
10.41
2All entities can now pass metadata through the pipeline. XML tasks can add these as additional attributes on output.
3Added --convert-change-to-full-history task.
4Added "initialize" support to all tasks to allow pre-processing setup to occur.
5Added a number of unit tests to increase test coverage, and reduced use of duplicated support classes.
6Added support for the XML bounds element, but still support old bound element.
7Added configurable log message prefix to the output of --log-progress-* tasks.
8Added completion step timings for --log-progress-* tasks.
9Added continuous replication looping support to apidb replication functionality.
10Added --write-replication task.
11Enhanced --replicate-apidb to send output through the pipeline to tasks like --write-replication instead of being an all-in-one task.
12Added --send-replication-sequence task which notifies listeners when new replication data has been written.
13Added --send-replication-data task which provides streaming replication data to listeners.
14Added --receive-replication task which consumes data sent by --serve-replication-data.
15Added --replication-to-change task which converts a replication pipeline stream to a normal change stream.
16Added keyValueListFile option to --node-key-value and --way-key-value tasks.
17Added JaCoCo code coverage tool to the build process.
18Updated ant scripts to use build, checkstyle, test and all targets across all projects.
19Update pgsnapshot and pgsimple tasks to no longer use PostGIS deprecated function names.
20Added --read-pbf-fast PBF reader which utilises multi-threading to improve performance.
21Enhanced XML tasks to handle missing lat/lon attributes on deleted entities.
22
230.40.1
24Fix the classworlds version in the Windows launcher.
25
260.40
27Fixed base64 encoding of user credentials sent by --upload-xml-change.
28Added support for merging Bound objects.
29Modified entity object creation to require use of CommonEntityData to simplify constructors.
30Added metatag facility to all entities to allow custom data to be passed through the pipeline.
31Added read timeout to replication retrieval code to prevent it blocking forever.
32Improved JPF loading code to avoid NullPointerException if encountering malformed plugins.
33Added ability to set input buffer size for all tasks using an input buffer.
34Relaxed Bound argument validation to allow points on all edges of the map to be included in bounding boxes.
35Modified PostGIS function calls to be PostGIS 2.0 compatible.
36Updated build scripts to support GIT.
37Do not emit bound entities when reading PBF files with no bbox information set.
38Add --compute-bounding-box and --set-bounding-box tasks for manipulating Bound entities.
39Added total object count display to --report-entity task.
40Updated all jar dependencies with exception of Xerces to latest versions.
41Modified project layouts to match Maven standard layout.
42Enhanced tests to dynamically generate test data instead of relying on ant-based data generation.
43Updated apidb tasks to support new rails3-based apidb schema (rename of id columns).
44Fixed name of --log-progress-change-0.6 task. Was registered as --log-change-progress-0.6.
45Modified Ivy to symlink to jar files on supported platforms rather copy jars into projects.
46
470.39
48Added the pgsimp tasks which are the old 0.36 version pgsql tasks supporting the old style separate tags tables.
49Added a new "cascadingRelations" flag to --bb and --bp which is less costly than "completeRelations" and makes sure that parent relations are emitted when a child relation of theirs is emitted even if that child happens to come after the parent in the stream. The list of relations on output may appear in a different order than on input.
50Fixed connection failure error handling in pgsql tasks to display the correct error message instead of a NullPointerException.
51Fixed serialisation logic to correctly identify and use the OsmUser.NONE singleton object.
52Fixed checkstyle errors in the PBF code.
53Renamed the PBF project to "pbf" instead of "binary".
54Improved performance in the DataPostbox class used for inter-thread messaging.
55Fixed PBF code to allow an empty file to be written.
56Updated PBF library dependency from version 1.0 to 1.1.
57Updated default PBF filename from dump.osmbin to dump.osm.pbf.
58Fixed PBF code to write a header block even if no bound entity exists.
59Added java code for PostgreSQL HStore support instead of only including a pre-compiled jar.
60Added a new "auto" option for the compressionMethod argument of XML tasks.
61Fixed the windows launch batch script to support installation to a different drive than the drive of the working directory.
62Fixed the UNIX launch shell script to support spaces in argument values.
63Enhanced --used-node to include nodes referenced by ways and relations, instead of just ways.
64Enhanced serialisation logic to support negative version numbers. Note this increases the space used by serialisation slightly.
65Added a --used-way task.
66
670.38
68Updated internal storage to use long instead of int identifiers to remove 2^31 node id limit.
69Updated apidb support to the latest schema 20100910084426.
70Fixed the --write-pbf short option --wb to point to the writer implementation instead of the reader.
71Added additional quoting to commands in the osmosis.bat launcher to fix issues with spaces in the installation path.
72Modified plugin loader code to support multiple osmosis-plugins.conf files to allow each plugin to provide its own.
73
740.37
75Updated the pgsql schema (now version 6) to move all tags into hstore columns, add a ways.nodes column and CLUSTER the nodes and ways tables. Significant performance increases.
76Fixed a bug in --bounding-box and --bounding-polygon preventing use of clipIncompleteEntities after a --tee task.
77Added --read-pbf and --write-pbf tasks supporting the new Google Protocol Buffer based binary data format.
78Added a new --flatten task for removing multiple versions of single entities from an entity stream.
79Fixed bugs in the --bounding-box and --bounding-polygon idTrackerType=Dynamic option.
80Enhanced the apidb replication read queries by adding temp table primary keys to fix performance regressions with PostgreSQL 8.4.
81
820.36
83Removed 0.5 support.
84Added the --fast-write-pgsql task which uses PostgreSQL COPY commands to insert data.
85Added the CompactTempFile node location store type which allows efficient pgsql geometry building for small data sets.
86All write-pgsql tasks now default to the CompactTempFile node location type store instead of in memory.
87Fixed regression in tar builds to make the launch scripts executable again.
88Updated the pgsql schema (now version 5) to add a primary key to the relation_members table to allow more efficient updates.
89Updated apidb support to the latest schema 20100513171259.
90Updated pgsql dataset bounding box queries to force more efficient query plans by disabling inefficient join types.
91Added a new dynamic idTrackerType implementation selected by default to the area filtering tasks to simplify usage.
92Added the --read-replication-lag task which indicates how far behind the server a local replication is.
93
940.35
95Registered --fast-read-xml as a task instead of requiring --fast-read-xml-0.6 to be specified.
96Fixed the completeRelations option on the --bounding-box and --bounding-polygon tasks to work as expected and documented.
97Updated the --fast-read-xml task to ignore changeset elements.
98Various enhancements to the Ivy processing including publishing of all artefacts.
99Fixed apidb tasks to correctly select the MySQL drivers instead of always using PostgreSQL drivers.
100Fixed the osmosis.bat launcher to handle spaces in the installation path.
101
1020.34
103Fixed the --sort and --sort-change tasks to use 0.6 task implementations instead of the older 0.5 implementations.
104Fixed the archive distributions to include the config/plexus.conf file, and include the launcher scripts in the tar version.
105Various cleanups of archive distributions to remove unnecessary files and minimise size.
106
1070.33
108Use Plexus Classworlds to simplify launcher scripts.
109Switch from org.apache.tools.bzip2 to commons-compress (this project is still alive).
110Updated the --apply-change task to detect full history diffs and abort.
111Enabled cascading deletes in the --truncate-apidb task to minimise foreign key violations on new schema updates.
112Updated all PostgreSQL database tasks to support IDENT authentication.
113Removed local postgis distribution since it is available on repo1.maven.org now.
114
1150.32
116Change to Public Domain licence.
117Re-factored all entity classes to use a CommonEntityData class to avoid duplication of code and simplify code re-use.
118Significant re-factoring of many apidb tasks to use Spring JDBC Template functionality.
119Added a --replicate-apidb task for extracting replication files based on PostgreSQL transaction ids.
120Fixed PostgreSQL apidb COPY files to include changeset id.
121Fixed the Windows launcher batch file to contain all libraries.
122Added a --read-replication-interval task.
123Updated XML writing tasks to ignore UTF-8 characters not supported by XML.
124Included new Xerces XML library to provide proper support for UTF-16 surrogate pairs.
125Added a --merge-replication-files task.
126Added a --way-key task.
127Added a --simplify-change task.
128Added pom.xml for maven artifacts
129
1300.31
131Added a --remove-tags task.
132Added JPF plugin support.
133Updated the build scripts to use Ivy dependency management.
134Added mutable entity support.
135Added writeable dataset support.
136Added apidb tasks that support both mysql and postgresql api databases.
137Added a new --append-change task for combining full history changesets.
138
1390.30
140Added 0.6 support.
141Enhanced the pgsql tasks and schema to support an optional "action" table.
142Enhanced the pgsql schema to introduce a stored procedure called during every changeset application.
143Enhanced the pgsql schema to include optional bbox and linestring columns on the way table.
144Added ability to specify slipping map zoom and coordinates as arguments to the bounding box filter.
145Added an "inMemoryBbox" to the pgsql writer task allowing it to calculate bounding boxes using a memory mapped file.
146Added an automatic VACUUM ANALYZE of the database for pgsql writer and truncation tasks.
147Added migration tasks --migrate and --migrate-change allowing 0.5 data to be converted to 0.6 format.
148Added forceUtf8 and profileSql options to all MySQL tasks.
149Added a --tag-sort task allowing tags within entities to be sorted alphabetically (0.6 only).
150Enhanced the buffer tasks to chunk objects passed (chunks are 1/4 of max buffer size) to minimise thread synchronisation (0.6 only).
151Dropped the default buffer size from 100 to 20 which provides slightly improved performance in most cases (0.6 only).
152Added the ability to generate full history changesets, as opposed to changesets with a single change per entity (0.6 only).
153Added a new allowIncorrectSchemaVersion option to most database tasks.
154Added new clipIncompleteEntities option to area filter tasks. It defaults to false which changes existing behaviour.
155
1560.29
157Fixed the --write-pgsql-simple-dump task to properly escape '\' characters.
158Updated the pgsql simple tasks to use an SRID of 4326.
159Updated build script to include a tar file distribution preserving launch script execute permissions.
160Enhanced the Entity class to allow dates to be represented as strings to avoid parsing overhead. Updated xml tasks to utilise this feature.
161Updated the pgsql simple tasks to use the new database schema which is more similar to the one used in the central MySQL
162Removed the old postgresql tasks.
163Removed "simple" from the names of the simple pgsql tasks.
164Added plugin support allowing new tasks in external jar files to be registered with osmosis.
165Added command line checks to ensure all task and option arguments are recognised, any misspelt arguments will now raise an error.
166Removed all static methods from the task registrar and task manager factory classes to avoid global data and simplify testing.
167Added the --node-key and --node-key-value tasks.
168
1690.28
170Added a maximum file download count feature to the change downloader to limit processing overhead when synchronising long time intervals.
171Fixed a major error in the TaskRegistrar from 0.27 where multiple tasks had the same name preventing osmosis from launching.
172
1730.27
174Added change downloader tasks providing the basis for automatic synchronisation of data sets.
175
1760.26
177Removed the bdb dataset implementation.
178Fixed an incorrect constant bug in the mysql tile calculator causing incorrect tile values to be generated.
179
1800.25
181Added a new osmosis-extract-mysql application for extracting regular changesets from mysql.
182Removed use of String.isEmpty to minimise Java5 incompatibility.
183Modified the default pipe connectivity functionality to use stack-based connectivity instead of queue-based connectivity.
184Added a populateCurrentTables option to the --write-mysql-change task to match the --write-mysql task.
185Added validation code to the merge tasks to verify that input data is sorted.
186Enhanced serialisation code to eliminate class identification information in the stream where single data types are being stored.
187Fixed a bug in the ListIdTracker class where unsorted ids were not being sorted correctly.
188Command line applications now returns status codes, 0 for success, non-zero for failure.
189Fixed the --read-api short name to be --ra instead of --wa.
190Updated store implementations to use buffered io providing significant performance improvements.
191Added a new osmosis-test-users-for-utf8 application for verifying the data integrity of the production user table.
192Added "bound" support to the pipeline.
193Added "-" alias for stdin/stdout to file reading/writing tasks for Cygwin & Windows
194Added a new dataset task --write-bdb for writing to a Berkeley DB implementation of a dataset.
195Added a new dataset task --read-bdb for reading from a Berkeley DB implementation of a dataset.
196Added a new dataset task --write-customdb for writing to a custom implementation of a dataset.
197Added a new dataset task --read-customdb for reading from a custom implementation of a dataset.
198Added a new dataset task --write-pgsql-simple for writing to a PostgreSQL implementation of a dataset.
199Added a new dataset task --write-pgsql-simple-dump for writing "COPY" load files for a PostgreSQL implementation of a dataset.
200Added a new dataset task --truncate-pgsql-simple for truncating all PostgreSQL dataset tables.
201Added a new dataset task --read-pgsql-simple for reading from a PostgreSQL implementation of a dataset.
202Added a new dataset task --dataset-bounding-box for extracting a bounding box from a dataset.
203Added a new dataset task --dataset-dump for dumping the entire contents of a dataset.
204Added new --used-node and --way-key-value tasks for filtering ways and their associated nodes with specific tags.
205
2060.24
207Fixed a bug in the XML node writing code where it would write a ',' character instead of '.' as the decimal separator in some locales.
208
2090.23
210Added support for a custom decoder when writing xml files to workaround the current production utf8 double encoding issue.
211Added short task names to shorten command line (eg. --write-xml becomes --wx).
212Fixed a node changeset bug where nodes created at the end of the interval were being excluded from the query.
213Added support for default task arguments requiring no name and added them for commonly used tasks requiring a single argument.
214
2150.22
216Updated mysql writing tasks to check if a timestamp is set and raise a meaningful error indicating where the error occurred.
217Modified mysql tasks to use the default character encoding for database connections to avoid suspected double encoding with the production database.
218Fixed the Entity serialisation to cater for null user ids and timestamps.
219Removed all 0.4 tasks.
220Added a new IdList id tracker implementation to be used internally by area filtering tasks to reduce memory usage on small'ish areas. Old BitSet implementation is still selectable if large bounding boxes are required.
221Added automatic compression method detection for all xml file tasks to avoid need to use compressionMethod argument.
222
2230.21
224Introduced a custom serialisation mechanism to improve compressed temporary file performance by approximately 5 times.
225Added a new RandomAccessObjectStore class which can be used as the basis for smarter tasks requiring access to data multiple times.
226Added completeWays and completeRelations arguments to the area filtering tasks to provide more control over the filtering process.
227Added new --tee and --tee-change tasks for sending data to multiple output destinations.
228
2290.20
230Fixed the integrity reporter task so that it writes new lines after each record.
231Relaxed validation checks in the --apply-change task so that it can apply changes to planets already containing data without aborting.
232
2330.19
234Added a new --integrity-reporter task to detect referential integrity issues between entity types.
235Updated the default file names for report tasks.
236Added a new --log-progress task to provide progress information to the jdk log during processing.
237Added a new --log-change-progress task to provide change progress information to the jdk log during processing.
238Fixed a number of bugs in the area extraction tasks.
239Updated area extraction tasks to support negative identifiers used on files that are ready for upload.
240Fixed bug in the ChangeWriter where the node insert SQL commands were missing tile parameters.
241
2420.18
243Improved error messages when errors are returned from the api.
244Added support for new shorter date format excluding millisecond information.
245Updated mysql writing code to support the new 0.5 schema version and fixed bugs in node insertion sql.
246Added support for quiet and verbose command line options.
247Modified command line arguments to default to using 0.5 tasks.
248
2490.17
250Added schema version checking to all MySQL tasks.
251Updated xml reading tasks to provide line number information when parse errors occur.
252Modified xml writing tasks to include the osmosis version number in the generator attribute.
253Updated mysql tasks to support the new tile indexing and fixed precision lat/lon.
254
2550.16
256Fixed a major bug in UTF-8 handling when writing xml files. All files were being written in the default character set.
257
2580.15
259Added full 0.5 support.
260Updated 0.4 code to the 005 schema.
261
2620.14
263Fixed a bug in the area filtering tasks where the original way was being passed to the destination instead of the modified way with only the available segments populated.
264Fixed a bug in the way comparison code where changes in the ordering of segments wouldn't be picked up.
265Fixed a bug in date parsing where the month was always being parsed a month in advance.
266Added the ability to load database login credentials from a properties file.
267Modified xml file writing to use 2 space indenting instead of 4.
268
2690.13
270Modified the --bounding-polygon task so it can read files containing blank lines.
271Added the ability to dump mysql "current" tables.
272Modified xml output to only write latitude and longitude to 7 decimal places.
273
2740.12
275Added a --read-api task for retrieving data from the central server.
276Added "user" support to the core data types, and all relevant tasks except database writing tasks.
277Added a --bounding-polygon task for filtering data by an area.
278Added a --report task which will produce user statistics for a data set.
279
2800.11
281Modified mysql snapshot task to read entire contents of table and process data in code in attempt to improve dump times.
282Added gzip and bzip2 support to xml tasks.
283Modified all database snapshot tasks to include entities with a timestamp equalling the snapshot time.
284Modified all database changeset tasks to not include the begin time and to include the end time.
285
2860.10
287Added JOSM-like date handling as a fallback if the standard xml date format isn't found.
288Added a custom date parser for the special case of GMT times which this tool always produces (approx 10x speedup over JOSM code and 100x speedup over standard JDK for GMT dates).
289
2900.9
291Fixed the name of the main class attribute in the jar manifest.
292Fixed a potential bug in the change application code where a validation step may have erroneously thrown errors.
293Added merge tasks to combine multiple entity or change files together.
294
2950.8
296Modified node and segment readers to avoid returning duplicate records where multiple records with identical timestamps exist.
297Moved main osmosis code into a "core" sub-package allowing new tools to be added.
298Fixed timeouts in mysql dumping of ways by using temporary files for query results.
299Refactored all temporary file handling to use common persistent iterator functionality.
300Fixed some classes not marked as Serializable.
301Updated change application to use less strict validation to handle "re-creates" of entities.
302Updated xml writing tasks to always write in UTC format.
303
3040.7
305Made a number of changes to the mysql dump task to improve performance and reduce the likelihood of timeouts.
306
3070.6
308Modified ChangeDeriver task to set the timestamp of deleted records to the current time.
309Fixed all uses of java.sql.Timestamp to be converted to java.util.Date explicitly instead of utilising inheritance.
310Fixed bug in change writer where timestamps weren't being written for segments.
311Added UserIdManager functionality which creates a system "Osmosis" user in the database to assign all changes to.
312
3130.5
314Modified command line date format to avoid use of spaces.
315
3160.4
317Modified mysql dump task to produce consistent snapshots.
318Added the ability to skip "current" tables when importing to mysql.
319Optimised mysql change reading queries.
320
3210.3
322Added mysql table truncator task.
323Updated segment xml processing code to handle timestamps.
324Made xml date processing during xml reading optional to improve performance.
325Added buffer tasks to allow pipeline processing to be split across multiple threads.
326Change reading and writing from a mysql database is now working, at least for simple test cases.
327
3280.2
329Added Main-Class feature to the jar manifest simplifying program launch.
330Added a shell launch script to simplify running on unix/linux.
331Added a readme with basic installation instructions.
332Added xml date parsing support.
333Fixed database dump code to read the "current" tables.
334Added a task for reading changes from a database.
335Added a task for writing changes to a database.
336Added a task for truncating database tables.
337Improved mysql import performance by disabling indexes and locking tables.
338
3390.1
340Initial version.
Note: See TracBrowser for help on using the repository browser.