s434730/RandomSec
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,683 Commits 6 Branches 0 Tags 1.3 GiB
09e0f270e4
Commit Graph

1120 Commits

Author SHA1 Message Date
David Huynh
a90a9c724e Forgot to register blank down operation in operation registry previously. 
Added uniques GEL function for eliminating duplicates in an array.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1158 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-16 03:00:43 +00:00
David Huynh
fa816007a7 Fixed copy-and-paste string mistake in BlankDownOperation. 
Fixed minor bug in Row.isValueBlank that returns true for non-string values.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1157 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-16 02:16:41 +00:00
David Huynh
e61655506a Added new command to import QA results, so any reconciliation action that yields conflicting or uncertain opinions among reviewers can be examined inside Gridworks. 
Added new customized facets for checking QA results. 

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1156 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-13 16:26:33 +00:00
David Huynh
8f071ede31 Added command Transpose Cells in Rows into Columns (Issue 82). 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1147 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-08 06:30:30 +00:00
David Huynh
d1a66e2e63 Added JSON support in GEL. 
Added GEL functions: escape, parseJson, hasField.
Fixed bug in preference store: expression history was still not loaded properly.
Integers are now rendered without decimals in the expression preview dialogs.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1145 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-07 22:57:48 +00:00
David Huynh
e70f16025b Fixed bug introduced recently by changing the preference key of the expression history from "expressions" to "scripting.expressions". 
Added code in FileProjectManager for trying to recover projects in the workspace dir but are not recorded in the workspace json file.


git-svn-id: http://google-refine.googlecode.com/svn/trunk@1144 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-07 20:25:31 +00:00
David Huynh
0500d7aa10 Added commands Move Column to Beginning, Move Column to End, Move Column Left, Move Column Right. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1142 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-07 01:24:48 +00:00
David Huynh
f0eae04c0c Forgot to add 2 files in the last commit 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1141 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-06 23:50:22 +00:00
David Huynh
a8ee9b9e08 Added Fill Down and Blank Down commands. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1140 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-06 20:33:28 +00:00
David Huynh
3bda9d035d Added support for creating a project by pointing to a data file URL. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1139 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-06 06:15:05 +00:00
David Huynh
f411dc9104 - Issue 112: Refactor Importer API (patch from tfmorris) 
- Added support for storing custom metadata in ProjectMetadata.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1138 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-06 05:04:25 +00:00
David Huynh
00c6865d95 - Select All and Unselect All buttons in History Extract dialog 
- Schema skeleton: support for multiple cells per cell-as nodes, and for conditional links


git-svn-id: http://google-refine.googlecode.com/svn/trunk@1137 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-05 20:27:39 +00:00
David Huynh
5cb3f924f6 Added support in protograph for specifying several column names per cell-as nodes. 
Started to add support for conditional links in protograph. The UI is not hooked up with.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1136 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-05 08:29:34 +00:00
David Huynh
b8ad56c6db Made sure in the schema skeleton dialog, in the dialog box for a node, in the "cell-as-topic" section, the type is always recorded. 
In the triple loader transposed node factory, use the column's recon config to generate new topics' type.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1135 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-05 00:53:08 +00:00
David Huynh
dcc3ac8534 Renamed packages com.metaweb.* to com.google.*. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1130 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-03 23:01:18 +00:00
Stefano Mazzocchi
8c56b437fa more fixes 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1129 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-03 20:19:48 +00:00
David Huynh
762a9f13eb Text facet's choice count limit is now configurable through preference page. Preference page needs polishing. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1127 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-02 01:49:10 +00:00
David Huynh
965ef20790 Made sure commands that create new columns check for duplicate column names. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1126 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-01 04:44:21 +00:00
David Huynh
4ad31ffcde Excel importer now supports "header lines" parameter. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1125 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-01 04:22:45 +00:00
David Huynh
7bb6674e5b Fixed recently introduced bug: expressions were not logged because preference stores were not initialized properly. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1124 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-31 07:01:44 +00:00
David Huynh
f069780bfa Added support for bundling .js files to shave off some loading time. 
For GetRowsCommand, tried to use jsonp but that didn't seem to improve performance much.
Gzip http responses of various text-based mime types.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1122 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-31 06:51:11 +00:00
David Huynh
d71d84194f Register new operation Transpose Cells in Columns into Rows. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1112 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-30 02:28:33 +00:00
David Huynh
ee14955605 Added new command Transpose Cells in Columns into Rows. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1111 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-30 02:25:58 +00:00
David Huynh
a192674118 - added smartSplit GEL function that can handle quoted values 
- added max width to operation extract dialog
- made GEL get and slice functions handle HasFieldsList
- fixed versioned standard-reconcile URLs (they need userid.user.dev)

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1110 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-26 22:50:00 +00:00
David Huynh
2ff0184c65 - switched to accessing versioned standard-reconcile app 
- standardized preference keys to using dot separated format
- added support to override freeq url from workspace preferences
- added GEL controls: forEachIndex, forRange, filter
- enforced max-width on preview table columns in expression preview dialog
- added preservedAllTokens option to split GEL function

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1109 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-26 21:12:40 +00:00
David Huynh
4522b98f32 Store and use job ID to retrieve MDO ID and send that in subsequent loads. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1100 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-16 17:32:06 +00:00
David Huynh
4373e7276f Pass target Freebase type IDs in recon objects to freeq. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1099 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-15 21:45:17 +00:00
David Huynh
b854f99ef5 Removed extra closing brace. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1096 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-11 05:54:06 +00:00
David Huynh
43dadf40da Added ignore:true to any triple that shouldn't be loaded. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1095 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-10 05:07:24 +00:00
David Huynh
513283d4d1 Support creation of cache directories, so the rdf importer can store its lucene indexes. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1090 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-08 23:22:29 +00:00
David Huynh
f5fc44e24e Refactoring to expose extension points that the rdf-exporter extension will plug into. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1074 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-06 00:14:07 +00:00
David Huynh
ab82562016 Tripleloader protograph transposer now generates more context information for QA. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1073 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-03 01:39:14 +00:00
David Huynh
217fb7b25c Fixed Issue 66: Records not excluded with inverted text facet. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1064 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-01 20:26:54 +00:00
Stefano Mazzocchi
a682d6b36f fixing eclipse warnings 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1063 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-01 08:03:42 +00:00
Stefano Mazzocchi
9fbff0640b make sure that splitting values maintains empty cells if the separator is repeated 
(this is useful in case the cells contains a rigid structure across multiple columns)

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1062 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-01 07:47:57 +00:00
Stefano Mazzocchi
2302d017d8 remove eclipse warnings 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1061 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-01 07:47:52 +00:00
David Huynh
18b720b913 Fixed CSV and TSV export bug. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1059 7d457c2a-affb-35e4-300a-418c747d4874
 2010-07-01 02:32:03 +00:00
David Huynh
2e3984d54a When transposing data to triple loader output, pass row indices and cell indices deep down so later we can generate more context information for recon. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1051 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-29 17:30:16 +00:00
David Huynh
0e4781cb58 Forgot a console.log() call. 
Allow reconciling against no particular type.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1043 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-27 04:20:35 +00:00
David Huynh
76c8cd77eb "search for match" links in data table cells now use recon service's entity suggest options. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1041 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-27 03:31:56 +00:00
David Huynh
ecfb893e98 More work on the recon UI. Standard services can now be added. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1038 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-26 01:10:23 +00:00
David Huynh
1342ceacea Careful not to load all projects in an autosave cycle. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1037 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-25 22:43:30 +00:00
David Huynh
058e86b4c8 First pass in trying to generalize standard reconciliation service UI. A lot of pieces are still Freebase-centric. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1032 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-24 20:13:51 +00:00
Iain Sproat
f0ed50e468 issue 69 fixed. ControlFunctionRegistry now correctly registers Chomp expression as "chomp" key. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1024 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-23 17:53:29 +00:00
David Huynh
a9f77d0f51 Minor bug. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1020 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-22 21:29:15 +00:00
Iain Sproat
0d7b3b0e9c ProjectManager is now partially unit tested. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1015 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-21 19:57:31 +00:00
Iain Sproat
dcf6919900 Functionality which didn't need to be moved to FileProjectManager as it wasn't file system specific has been moved back to ProjectManager. importProject function is now named loadProjectMetadata to avoid confusion. 
Some additional source code documentation added to ProjectManager, and methods rearranged in more readable fashion.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1011 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-20 16:32:49 +00:00
Iain Sproat
7ced0cb31e New feature for importing text files (CSV and TSV). By selecting the checkbox in index.html it allows the effects of quotation marks around data values to be ignored. 
Unit test added for this.

This has required a further branch to opencsv - patch sent to opencsv project and can be tracked at  https://sourceforge.net/tracker/?func=detail&aid=3018599&group_id=148905&atid=773543

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1010 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-20 14:47:45 +00:00
Iain Sproat
0af7e5fcf5 More functionality which didn't need to be moved to FileProjectManager, as it wasn't file system specific, has been moved back to ProjectManager. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@992 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-18 20:47:15 +00:00
Iain Sproat
c72b4571a5 Functionality which didn't need to be moved to FileProjectManager as it wasn't file system specific has been moved back to ProjectManager. 
Some additional source code documentation added.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@991 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-18 19:16:30 +00:00
Iain Sproat
846cf1d57e Fixed bug in CsvExporter, all unit tests for CsvExporter and TsvExporter now working. 
History now has the beginnings of a unit test.

Additional source documentation on public methods in ProjectManager and History.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@989 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-17 15:37:28 +00:00
David Huynh
e7d0fc5ed6 Implemented a generic preference store for both the whole workspace and each project. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@988 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-17 02:15:26 +00:00
Iain Sproat
18e319bb76 Moved call to FileHistoryEntryManager from ProjectManager to FileProjectManager. 
Added interface HistoryEntryManager, which seems to have been forgotten from last commit.
FileHistoryEntry is now named FileHistoryEntryManager.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@983 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-16 14:17:17 +00:00
Iain Sproat
f92fc2d056 Internal refactor for IO - HistoryEntry is now a concrete class, so can be instantiated (reverting Operations classes back to r972 which were changed as a result of HistoryEntry being abstract). 
HistoryEntry now deals with backend (filesystem etc.) through classes which implement HistoryEntryManager.  This HistoryEntryManager is held by ProjectManager, which allows for FileProjectManager to create FileHistoryEntryManager as appropriate.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@982 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-16 12:35:37 +00:00
Iain Sproat
280daad2f6 Refactored ImportProjectCommand and ExportProjectCommand. These are no longer dependent on the File System, and all file system related work is done in FileProjectManager. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@981 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-16 07:44:46 +00:00
Iain Sproat
f47cb75525 Fixed ImportProjectCommand so it no longer contains references to project.html, a file previously removed from the project. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@980 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-16 07:06:52 +00:00
Iain Sproat
17f1dc2e6f The file system coupled method getProjectDirectory is now removed from ProjectManager. 
Methods of HistoryEntry which directly work with the File System have been moved to FileHistoryEntry in the io directory, and HistoryEntry made abstract.

As the abstract HistoryEntry cannot be instantiated directly, the ProjectManager is now responsible for creating new HistoryEntry.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@973 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 22:11:35 +00:00
Iain Sproat
b07075bed5 FileProjectManager and portions of Project and ProjectMetadata classes which deal with io are moved to an io directory. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@972 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 20:55:38 +00:00
Iain Sproat
c94957b6a0 CreateProjectCommand no longer contains references to project.html, a file previously removed from the project. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@971 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 20:28:18 +00:00
Iain Sproat
dc7060d390 portion of ProjectManager which interacts with File System has been moved to FileProjectManager, which extends ProjectManager. ProjectManager is now abstract. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@970 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 19:34:40 +00:00
Iain Sproat
a671551289 Two more XmlImport tests now work. Some documentation stubs were added to XmlImporterUtilities. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@967 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 16:22:53 +00:00
David Huynh
f7fe44dccc Converted project.html to project.vt and added a client side resource manager, where extensions can register scripts and styles to be included in .vt files 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@965 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 00:35:23 +00:00
David Huynh
b0389d8c6a Jython integration has been moved out to an extension. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@964 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-14 22:11:34 +00:00
Stefano Mazzocchi
af48cb799e moving Griworks to use the Butterfly webapp framework (this will allow us to make gw more extensible without excessive complexity... as a bonus we gain server side javascript support which might end up being useful) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@940 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-05 00:50:18 +00:00
Stefano Mazzocchi
0648e8725e adding regexp group capturing GEL function 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@932 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-01 08:54:17 +00:00
Stefano Mazzocchi
5e0acf28d0 forgot to add the ngram class itself 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@931 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-01 08:04:40 +00:00
Stefano Mazzocchi
b3173211e3 adding an ngram function to GEL 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@930 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-01 08:02:28 +00:00
Stefano Mazzocchi
3b7f132430 fixing jython initialization logic 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@924 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-31 09:00:42 +00:00
Stefano Mazzocchi
e3fc7ab603 bringing the refactor branch up to speed with trunk 
(everything works like in trunk for now, although some tests still fail)


git-svn-id: http://google-refine.googlecode.com/svn/branches/split-refactor@915 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-30 18:18:59 +00:00
Stefano Mazzocchi
aa4de48f95 some renaming, moving tests into main 
git-svn-id: http://google-refine.googlecode.com/svn/branches/split-refactor@906 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-30 16:55:53 +00:00