David Huynh
67662fcc96
Escape strings from TSV exporter.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@485 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-16 06:45:57 +00:00
Stefano Mazzocchi
1e5a787281
avoid ArrayOutOfBoundsException
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@484 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-15 20:24:23 +00:00
Stefano Mazzocchi
397861b612
- replace the 'cos' library with the apache 'commons-fileupload' for licensing reason (the cos library had a weird arm-twisting license that forced you to buy an o'reilly book on servlets for each developer in your company... good thing I read it all)
...
- some tweaks on imgareaselect's look
git-svn-id: http://google-refine.googlecode.com/svn/trunk@483 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-15 18:42:41 +00:00
Stefano Mazzocchi
8cf69301a5
added a new command to get column metadata prior of creating the scatterplot half-matrix, this allows us to build a much more compact table and make the browser crawl a little less
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@481 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-15 06:20:56 +00:00
David Huynh
155b5a483a
When deleting project dirs, we need to recurse into them ourselves.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@480 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-15 00:17:08 +00:00
David Huynh
1d938bc4d0
Better MQL batching during extending data operations.
...
Tried to use JSON streaming in changes as well.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@479 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-15 00:09:14 +00:00
David Huynh
2277f45ef6
For jython, wrap native values properly using Py.java2py().
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@478 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-14 21:30:39 +00:00
David Huynh
24a7ea91b6
Fixed bugs
...
- MassEditOperation was barfing when engineConfig was missing
- When parsing JSON in streaming mode, get long instead of int and double instead of float so that we won't get overflow exception.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@476 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-14 20:34:29 +00:00
Stefano Mazzocchi
3bae823010
fixed eclipsed warning (no functional change)
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@473 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-14 02:39:24 +00:00
David Huynh
4a06c49a9a
Added streaming json parser for faster re-loading of existing projects.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@470 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 23:57:03 +00:00
David Huynh
a1a8758c37
Added options for specifying # lines the header columns take, and the # lines to skip processing entirely initially.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@468 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 21:23:41 +00:00
Stefano Mazzocchi
dc4b63d2bf
forgot a piece
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@465 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 07:27:20 +00:00
David Huynh
a2db5590ac
Trim column names on import.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@461 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 06:28:13 +00:00
Stefano Mazzocchi
e232a90a73
progress but still no worky on the scatterfacet
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@457 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 03:44:42 +00:00
Stefano Mazzocchi
ba85f50e39
adding log-log support to the scatterplot matrix and more controls
...
(the scatterfacet still doesn't work but this is already more useful)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@456 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-13 00:25:43 +00:00
David Huynh
8b95248c75
Fixed bug where after reconciling by ID, GUID, or key would generate a buggy numeric range facet, since all the scores were artificially the same.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@454 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-12 22:19:44 +00:00
Stefano Mazzocchi
7ab1acd801
skeleton code for scatterfacet
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@453 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-12 19:22:49 +00:00
David Huynh
8fb23913ce
Added "time" part option to datePart function.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@448 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-11 23:07:56 +00:00
David Huynh
ce8963d009
Added datePart function.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@447 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-11 23:01:34 +00:00
David Huynh
f7e830e709
Fixed bug in which editing a single cell and then starring the same row seemed to revert the cell back to its original content.
...
Added an option for not guessing cell value type during import.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@446 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-11 21:54:56 +00:00
David Huynh
75ea8304a3
In numeric range facets, show the other choices only if they have positive counts in the base distribution.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@445 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-11 08:19:05 +00:00
David Huynh
5928a689e2
Use RowParser for parsing the header row, too.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@444 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-11 03:42:44 +00:00
Stefano Mazzocchi
81fb2f1740
first step at scatterplot facet selector
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@442 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-10 08:28:06 +00:00
Stefano Mazzocchi
a286b805d5
no reason to process the contextual rows while clustering
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@440 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-10 08:11:31 +00:00
David Huynh
70449cf7c8
Better error catching in toNumber function.
...
Watch out for the string "Infinity" while importing data sets: don't parse it into a double.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@438 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-09 21:59:50 +00:00
David Huynh
8950e87e02
When re-loading existing projects from disk, cache recon objects by their IDs to lower memory consumption.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@437 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-09 19:32:50 +00:00
David Huynh
dff870519e
Made numeric range index handle value.log() properly when value is 0 or negative.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@436 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-09 05:24:46 +00:00
David Huynh
a0d8c385f9
Do a bit more checking when retrieving project metadata just in case project metadata is null.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@435 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-09 04:52:32 +00:00
Stefano Mazzocchi
d3d40d608a
bunch of PMD-induced fixes
...
(now the PMD report is clean)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@430 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-09 00:14:11 +00:00
David Huynh
759824e1b4
Bug fix: editing one facet choice while some other choices are selected resulted in no change.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@429 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-08 22:16:12 +00:00
Stefano Mazzocchi
7526c4e582
cleanups (no functional changes)
...
this makes pmd and javac on linux happier
git-svn-id: http://google-refine.googlecode.com/svn/trunk@427 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-08 20:46:02 +00:00
Will Moffat
efb9a43b73
Added: toDate() now supports formats.
...
My dataset has the same dates written in two styles: Nov-09 and 11/09
You can now parse this using: value.toDate2('MM/yy','MMM-yy').toString('yyyy-MM')
git-svn-id: http://google-refine.googlecode.com/svn/trunk@414 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-07 19:12:08 +00:00
David Huynh
0996b9e1dd
Gzip project export tar files.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@394 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 22:28:30 +00:00
David Huynh
5320cc6587
Make duplicated column names unique during import by appending indices to them.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@392 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 17:55:36 +00:00
David Huynh
9d9329ca96
Implemented row remove command.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@391 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 07:47:44 +00:00
David Huynh
1fd85c62bf
Implemented column rename command.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@390 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 07:15:34 +00:00
David Huynh
93d6f9fc54
Better error message for bad regular expressions in GEL.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@388 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 06:18:07 +00:00
David Huynh
f402db10af
Implemented inter-project joins.
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@387 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-06 05:35:48 +00:00
Stefano Mazzocchi
771810bc0d
avoid exception if there is only one extension in the whole archive
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@385 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-05 21:36:27 +00:00
Stefano Mazzocchi
2efbf0031f
- removed the 'thirdparty' directory (now the 'gridworks' script will download and install needed tools if they are not present in the system already)
...
- added 'findbugs' command that uses the findbugs static analyzer to look for problems in the code
- fixed a bunch of issues that findbugs found (a few methods would go a little faster, and a few NPE will be avoided... nothing major but good to have)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@382 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-05 07:15:16 +00:00
Stefano Mazzocchi
798b2a36ca
- archive and compressed file importer (supports zip, tar, gz, bz2, tar.gz and tar.bz2)
...
(works by loading the files that have the most common extensions in the archive)
- changed default max heap to 3Gb
git-svn-id: http://google-refine.googlecode.com/svn/trunk@381 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-04 07:48:47 +00:00
Stefano Mazzocchi
c24ec94835
had to shuffle around a bunch of classes to separate the main server classloader from the context classloader and allow reloading to happen for real
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@377 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-03 01:01:11 +00:00
Stefano Mazzocchi
72203cd3d5
- moved all code that contained MIT IP outside ( http://code.google.com/p/simile-vicino/ )
...
- moved bzip2 and tar code from apache ant into their own jar files
- now gridworks source contains only com.metaweb.* code everything else is a jar dependency
- started to work on archive importer
git-svn-id: http://google-refine.googlecode.com/svn/trunk@376 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-02 23:40:12 +00:00
Stefano Mazzocchi
4eda7ae2c0
avoid an array out of bounds exception in case there are no columns in the dataset
...
(I know, it should not happen but when it does let's not barf)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@375 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-02 20:21:41 +00:00
Stefano Mazzocchi
62f5f21ca3
atom is handled as well by the XML importer
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@374 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-02 06:44:05 +00:00
Stefano Mazzocchi
83faee3aa9
add a frame-less menu item in macosx to be able to open another gridworks browser window/tab in case we closed it by mistake
...
(no idea how to do this on windows, though, since there is no frame-less menu concept there)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@373 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-02 06:39:41 +00:00
Stefano Mazzocchi
521acda025
- pass the svn revision as format version (for more detailed verification)
...
- add an 'autoreload' setting that makes Gridworks autoreload its self if a class gets changed
(this is useful to make development cycles faster when working on the java code with autocompiling IDE like Eclipse or IDEA)
git-svn-id: http://google-refine.googlecode.com/svn/trunk@372 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-02 00:52:38 +00:00
Stefano Mazzocchi
988378c761
Hmm, String.split() bites us again: use the commons-lang one instead to avoid having to escape regexp values (this was preventing a user from splitting by "." in GEL)
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@370 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-01 17:49:31 +00:00
Stefano Mazzocchi
0e07ec7acc
crude, I know, but for now make Gridworks digest RDF/XML as it was XML (works surprisingly well, btw)
...
git-svn-id: http://google-refine.googlecode.com/svn/trunk@369 7d457c2a-affb-35e4-300a-418c747d4874
2010-04-01 16:56:38 +00:00
Stefano Mazzocchi
dced641599
- added the ability to specify the character separator for CSV or TSV files that don't use commas or tabs (this was needed to parse a dataset that we got from the BBC to try things out)
...
- used commons-lang split function instead of the java String.split one, this is necessary to avoid having to escape separators that might be confused for regexps
git-svn-id: http://google-refine.googlecode.com/svn/trunk@368 7d457c2a-affb-35e4-300a-418c747d4874
2010-03-31 22:34:21 +00:00