s434730/RandomSec
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
471 Commits 6 Branches 0 Tags 1.3 GiB
6be29f712c
Commit Graph

267 Commits

Author SHA1 Message Date
David Huynh
24a7ea91b6 Fixed bugs 
- MassEditOperation was barfing when engineConfig was missing
- When parsing JSON in streaming mode, get long instead of int and double instead of float so that we won't get overflow exception.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@476 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-14 20:34:29 +00:00
Stefano Mazzocchi
3bae823010 fixed eclipsed warning (no functional change) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@473 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-14 02:39:24 +00:00
David Huynh
4a06c49a9a Added streaming json parser for faster re-loading of existing projects. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@470 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 23:57:03 +00:00
David Huynh
a1a8758c37 Added options for specifying # lines the header columns take, and the # lines to skip processing entirely initially. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@468 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 21:23:41 +00:00
Stefano Mazzocchi
dc4b63d2bf forgot a piece 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@465 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 07:27:20 +00:00
David Huynh
a2db5590ac Trim column names on import. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@461 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 06:28:13 +00:00
Stefano Mazzocchi
e232a90a73 progress but still no worky on the scatterfacet 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@457 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 03:44:42 +00:00
Stefano Mazzocchi
ba85f50e39 adding log-log support to the scatterplot matrix and more controls 
(the scatterfacet still doesn't work but this is already more useful)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@456 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-13 00:25:43 +00:00
David Huynh
8b95248c75 Fixed bug where after reconciling by ID, GUID, or key would generate a buggy numeric range facet, since all the scores were artificially the same. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@454 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-12 22:19:44 +00:00
Stefano Mazzocchi
7ab1acd801 skeleton code for scatterfacet 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@453 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-12 19:22:49 +00:00
David Huynh
8fb23913ce Added "time" part option to datePart function. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@448 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-11 23:07:56 +00:00
David Huynh
ce8963d009 Added datePart function. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@447 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-11 23:01:34 +00:00
David Huynh
f7e830e709 Fixed bug in which editing a single cell and then starring the same row seemed to revert the cell back to its original content. 
Added an option for not guessing cell value type during import.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@446 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-11 21:54:56 +00:00
David Huynh
75ea8304a3 In numeric range facets, show the other choices only if they have positive counts in the base distribution. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@445 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-11 08:19:05 +00:00
David Huynh
5928a689e2 Use RowParser for parsing the header row, too. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@444 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-11 03:42:44 +00:00
Stefano Mazzocchi
81fb2f1740 first step at scatterplot facet selector 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@442 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-10 08:28:06 +00:00
Stefano Mazzocchi
a286b805d5 no reason to process the contextual rows while clustering 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@440 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-10 08:11:31 +00:00
David Huynh
70449cf7c8 Better error catching in toNumber function. 
Watch out for the string "Infinity" while importing data sets: don't parse it into a double.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@438 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-09 21:59:50 +00:00
David Huynh
8950e87e02 When re-loading existing projects from disk, cache recon objects by their IDs to lower memory consumption. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@437 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-09 19:32:50 +00:00
David Huynh
dff870519e Made numeric range index handle value.log() properly when value is 0 or negative. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@436 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-09 05:24:46 +00:00
David Huynh
a0d8c385f9 Do a bit more checking when retrieving project metadata just in case project metadata is null. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@435 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-09 04:52:32 +00:00
Stefano Mazzocchi
d3d40d608a bunch of PMD-induced fixes 
(now the PMD report is clean)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@430 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-09 00:14:11 +00:00
David Huynh
759824e1b4 Bug fix: editing one facet choice while some other choices are selected resulted in no change. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@429 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-08 22:16:12 +00:00
Stefano Mazzocchi
7526c4e582 cleanups (no functional changes) 
this makes pmd and javac on linux happier


git-svn-id: http://google-refine.googlecode.com/svn/trunk@427 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-08 20:46:02 +00:00
Will Moffat
efb9a43b73 Added: toDate() now supports formats. 
My dataset has the same dates written in two styles: Nov-09 and 11/09
You can now parse this using: value.toDate2('MM/yy','MMM-yy').toString('yyyy-MM')



git-svn-id: http://google-refine.googlecode.com/svn/trunk@414 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-07 19:12:08 +00:00
David Huynh
0996b9e1dd Gzip project export tar files. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@394 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 22:28:30 +00:00
David Huynh
5320cc6587 Make duplicated column names unique during import by appending indices to them. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@392 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 17:55:36 +00:00
David Huynh
9d9329ca96 Implemented row remove command. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@391 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 07:47:44 +00:00
David Huynh
1fd85c62bf Implemented column rename command. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@390 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 07:15:34 +00:00
David Huynh
93d6f9fc54 Better error message for bad regular expressions in GEL. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@388 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 06:18:07 +00:00
David Huynh
f402db10af Implemented inter-project joins. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@387 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-06 05:35:48 +00:00
Stefano Mazzocchi
771810bc0d avoid exception if there is only one extension in the whole archive 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@385 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-05 21:36:27 +00:00
Stefano Mazzocchi
2efbf0031f - removed the 'thirdparty' directory (now the 'gridworks' script will download and install needed tools if they are not present in the system already) 
- added 'findbugs' command that uses the findbugs static analyzer to look for problems in the code
- fixed a bunch of issues that findbugs found (a few methods would go a little faster, and a few NPE will be avoided... nothing major but good to have)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@382 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-05 07:15:16 +00:00
Stefano Mazzocchi
798b2a36ca - archive and compressed file importer (supports zip, tar, gz, bz2, tar.gz and tar.bz2) 
(works by loading the files that have the most common extensions in the archive)
- changed default max heap to 3Gb


git-svn-id: http://google-refine.googlecode.com/svn/trunk@381 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-04 07:48:47 +00:00
Stefano Mazzocchi
c24ec94835 had to shuffle around a bunch of classes to separate the main server classloader from the context classloader and allow reloading to happen for real 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@377 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-03 01:01:11 +00:00
Stefano Mazzocchi
72203cd3d5 - moved all code that contained MIT IP outside (http://code.google.com/p/simile-vicino/) 
- moved bzip2 and tar code from apache ant into their own jar files
- now gridworks source contains only com.metaweb.* code everything else is a jar dependency
- started to work on archive importer


git-svn-id: http://google-refine.googlecode.com/svn/trunk@376 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-02 23:40:12 +00:00
Stefano Mazzocchi
4eda7ae2c0 avoid an array out of bounds exception in case there are no columns in the dataset 
(I know, it should not happen but when it does let's not barf)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@375 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-02 20:21:41 +00:00
Stefano Mazzocchi
62f5f21ca3 atom is handled as well by the XML importer 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@374 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-02 06:44:05 +00:00
Stefano Mazzocchi
83faee3aa9 add a frame-less menu item in macosx to be able to open another gridworks browser window/tab in case we closed it by mistake 
(no idea how to do this on windows, though, since there is no frame-less menu concept there)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@373 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-02 06:39:41 +00:00
Stefano Mazzocchi
521acda025 - pass the svn revision as format version (for more detailed verification) 
- add an 'autoreload' setting that makes Gridworks autoreload its self if a class gets changed
(this is useful to make development cycles faster when working on the java code with autocompiling IDE like Eclipse or IDEA)


git-svn-id: http://google-refine.googlecode.com/svn/trunk@372 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-02 00:52:38 +00:00
Stefano Mazzocchi
988378c761 Hmm, String.split() bites us again: use the commons-lang one instead to avoid having to escape regexp values (this was preventing a user from splitting by "." in GEL) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@370 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-01 17:49:31 +00:00
Stefano Mazzocchi
0e07ec7acc crude, I know, but for now make Gridworks digest RDF/XML as it was XML (works surprisingly well, btw) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@369 7d457c2a-affb-35e4-300a-418c747d4874
 2010-04-01 16:56:38 +00:00
Stefano Mazzocchi
dced641599 - added the ability to specify the character separator for CSV or TSV files that don't use commas or tabs (this was needed to parse a dataset that we got from the BBC to try things out) 
- used commons-lang split function instead of the java String.split one, this is necessary to avoid having to escape separators that might be confused for regexps


git-svn-id: http://google-refine.googlecode.com/svn/trunk@368 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-31 22:34:21 +00:00
Stefano Mazzocchi
7c132cfa53 clean eclipse warnings 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@357 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-29 20:51:18 +00:00
David Huynh
1d0e6abaf8 Got some work done on the plane: 
- better detection of record XML elements in XML importer
- XML importer creates column groups and data table view renders them


git-svn-id: http://google-refine.googlecode.com/svn/trunk@356 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-27 05:23:09 +00:00
David Huynh
2a9fbd7d81 Made sure columns are named hierarchically in XML importer. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@355 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-24 23:15:10 +00:00
David Huynh
df7389876f First shot at XML import. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@354 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-24 23:08:08 +00:00
David Huynh
4e76155652 Use application/x-unknown when exporting TSV so that the browser just saves the result. This is good for large exports, which overload the browser if the browser tries to display them. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@353 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-24 17:51:27 +00:00
David Huynh
47cad64a3f Properly unescape \t, \r, \n, \\. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@352 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-24 17:50:44 +00:00
David Huynh
30e3ca4965 Added splitByLengths function. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@351 7d457c2a-affb-35e4-300a-418c747d4874
 2010-03-24 04:30:31 +00:00