s434730/RandomSec
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
3,338 Commits 6 Branches 0 Tags 1.3 GiB
ea19f6dff1
Commit Graph

292 Commits

Author SHA1 Message Date
Iain Sproat
2f564589f5 Adding a Fixed Width data importer (Issue 85) and associated tests. 
Although this importer is 'wired up', it requires a property "fixed-column-widths" which is not (yet) implemented in the UI.  But the ImporterRegister.guessImporter method will probably select the CsvTsvImporter before the FixedWidthImporter anyway.  I suggest an improvement to the project creation UI and/or the guessImporter method will be required.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1857 7d457c2a-affb-35e4-300a-418c747d4874
 2010-11-11 13:15:41 +00:00
Iain Sproat
4f9dc38066 TsvCsvImporter now uses the CSV test methods to also test the TSV importer. Separate tests for the two importers are no longer required. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1856 7d457c2a-affb-35e4-300a-418c747d4874
 2010-11-11 13:06:39 +00:00
Iain Sproat
37a6f938c3 The TestNG DataProvider for the Tsv/Csv importer tests was previously providing just one test case with two parameters, rather than the two test cases of one parameter each which is required. This is now fixed, and the tests pass. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1855 7d457c2a-affb-35e4-300a-418c747d4874
 2010-11-11 10:52:18 +00:00
David Huynh
5a17acfd70 Prepended license text to java source 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1613 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-20 20:45:52 +00:00
Tom Morris
05d8180016 Disable failing test until it can be debugged 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1602 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-18 05:52:11 +00:00
Tom Morris
7dcd0c073d Revert bad commit r1600 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1601 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-18 05:19:05 +00:00
Tom Morris
79c00bab36 Incomplete - task 157: Integrate Google Spreadsheet import/export plugin 
http://code.google.com/p/google-refine/issues/detail?id=157

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1600 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-18 04:59:39 +00:00
Tom Morris
c8bb6a3475 Fix failing template exporter test. Disable HTML exporter no column headers test (unimplemented) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1570 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-15 18:07:17 +00:00
Tom Morris
aba75a9b4a Add a couple more tests 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1568 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-15 17:10:07 +00:00
Tom Morris
87b6d940b7 Some very basic tests for exporters with no test coverage 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1567 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-15 15:50:25 +00:00
Tom Morris
0041863a13 Placeholder (failing) tests for exporters with no test coverage 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1566 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-15 13:27:44 +00:00
David Huynh
73042712ed Made csv/tsv importer not trim whitespace even if "guess cells' types" is checked (for cells that are strings). 
Updated csv tests to expect un-trimmed cells.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1557 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-15 05:30:15 +00:00
Iain Sproat
d977f42f51 Changed behaviour of the XmlImporter to make it more permissive, and allow arrays within mixed elements to be used as candidates for importing to Refine. 
This change has also allowed the JsonImporter to pass all its unit tests without any further modification.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1425 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-04 18:33:59 +00:00
Iain Sproat
ec0581e3a9 Added two additional unit tests for ensuring the XmlImporter is behaving correctly. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1424 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-04 17:32:27 +00:00
Iain Sproat
8d51223610 An additional unit test for the JsonImporter, which it unfortunately fails. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1423 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-04 15:19:29 +00:00
Iain Sproat
ec9898ba92 Some tidying up of the XmlImporter which reduces the number of generic TreeParser tokens to a minimum - and should allow elements such as comments and CDATA to be ignored/skipped. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1422 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-04 15:02:09 +00:00
Iain Sproat
d3f223c196 The JsonImporter now passes all current unit tests. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1421 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-04 10:02:50 +00:00
Iain Sproat
2d47885182 Tests in ProjectManagerTests.java and RefineServletTests.java have had minor fixes to update them to the current code. 
doPostRegressionTest() still fails when run in the suite, but passes when run as an isolated test - there appears to be some cross-contamination from tests that I can't figure out atm.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1418 7d457c2a-affb-35e4-300a-418c747d4874
 2010-10-01 11:53:27 +00:00
Iain Sproat
d285999da8 New JsonImporter, JsonParser and JsonImporterTests (copy of XmlImporterTests with syntax of the example data altered for Json). 
Renaming of TreeImporter to TreeImportUtilities (as per the current convention with the XmlImporter and XmlImportUtilities).

NB the new JsonParser class does not work, and 5 of the new unit tests for JsonImporter currently fail.  To be fixed in due course.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1361 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-27 22:53:17 +00:00
Iain Sproat
e5ddfa6fdc All methods in XmlImportUtilities now use the TreeParser interface. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1323 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-27 17:59:53 +00:00
Iain Sproat
d71c563831 XmlImportUtilities.detectPathFromTag and XmlImportUtilities.detectRecordElement methods now use a generic TreeParser interface. A lightweight wrapper XmlParser wraps XMLStreamReader to provide parsing for xml data. 
This is another small step towards a generic importer for tree structured data.  My plan is to refactor more of XmlImportUtilities' methods to use the TreeParser interface so that XmlStreamReader is no longer called directly from XmlImportUtilities.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1322 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-27 17:40:51 +00:00
Iain Sproat
1bda46d40f Methods which are generic to any tree structured data and don't rely on an XmlParser have been moved to a new TreeImporter class. This is a small step towards supporting importers for other tree structured data. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1321 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-27 16:09:44 +00:00
Iain Sproat
2df0d1ac3a Eclipse .launch file for tests now correctly points to project 'grefine' rather than incorrect 'refine'. 
XmlImport tests now pass - the tests themselves were not testing the current xml being input to them.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1320 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-27 14:47:06 +00:00
David Huynh
d15fe661fa Renamed in client-side code. Refine should start and existing projects should still work. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1293 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-22 22:55:28 +00:00
David Huynh
1367ce301e More renaming, except for: client-side code, build scripts, anything to do with data loading and QA, workspace path. Refine can still run, and undo/redo on existing projects is working. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1290 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-22 18:36:33 +00:00
David Huynh
e6bc603a11 Renamed Java classes whose names contain 'Gridworks'. Refine is still able to start. But don't check out the code just yet. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1289 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-22 17:46:39 +00:00
David Huynh
edb23eb263 Changed Java packages com.google.gridworks.* to com.google.refine.* and modified other code just enough to start grefine up without error. Much remains to be done. Do not check out the code just yet. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1288 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-22 17:04:10 +00:00
David Huynh
2609c4049d Fixed issue 114: "Refactor project manager api to allow importers to create project metadata" by incorporating tfmorris' patch. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1271 7d457c2a-affb-35e4-300a-418c747d4874
 2010-09-17 01:00:23 +00:00
David Huynh
367796488e Fixed xml importer: subgroups should now line up properly by rows. 
Added command to reorder columns using drag and drop.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1227 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-25 06:17:08 +00:00
David Huynh
c94abd0427 Commands are now registered in association with their modules, so to avoid name collision. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1193 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-22 06:16:13 +00:00
David Huynh
dcc3ac8534 Renamed packages com.metaweb.* to com.google.*. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1130 7d457c2a-affb-35e4-300a-418c747d4874
 2010-08-03 23:01:18 +00:00
Iain Sproat
0d7b3b0e9c ProjectManager is now partially unit tested. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1015 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-21 19:57:31 +00:00
Iain Sproat
7ced0cb31e New feature for importing text files (CSV and TSV). By selecting the checkbox in index.html it allows the effects of quotation marks around data values to be ignored. 
Unit test added for this.

This has required a further branch to opencsv - patch sent to opencsv project and can be tracked at  https://sourceforge.net/tracker/?func=detail&aid=3018599&group_id=148905&atid=773543

git-svn-id: http://google-refine.googlecode.com/svn/trunk@1010 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-20 14:47:45 +00:00
Iain Sproat
846cf1d57e Fixed bug in CsvExporter, all unit tests for CsvExporter and TsvExporter now working. 
History now has the beginnings of a unit test.

Additional source documentation on public methods in ProjectManager and History.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@989 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-17 15:37:28 +00:00
Iain Sproat
a671551289 Two more XmlImport tests now work. Some documentation stubs were added to XmlImporterUtilities. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@967 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-15 16:22:53 +00:00
Stefano Mazzocchi
ef6f03694d make all tests pass 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@945 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-05 01:30:12 +00:00
Stefano Mazzocchi
82d3730568 no need for this anymore 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@927 7d457c2a-affb-35e4-300a-418c747d4874
 2010-06-01 06:56:54 +00:00
Stefano Mazzocchi
fe004d5fa6 fixed client tests 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@926 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-31 17:56:07 +00:00
Stefano Mazzocchi
7ce78776c6 refactored tests to avoid using static initialization which conflicts with our ability to have special log4j configurati 
ons for testings. Also tagged all broken tests as broken so that the tree is now green. Also used the test class as the
logger name so that it's easier to groupt the logger verbosity based on java packages.

git-svn-id: http://google-refine.googlecode.com/svn/trunk@922 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-31 08:51:51 +00:00
Stefano Mazzocchi
e3fc7ab603 bringing the refactor branch up to speed with trunk 
(everything works like in trunk for now, although some tests still fail)


git-svn-id: http://google-refine.googlecode.com/svn/branches/split-refactor@915 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-30 18:18:59 +00:00
Stefano Mazzocchi
8369961b9e renaming tests 
git-svn-id: http://google-refine.googlecode.com/svn/branches/split-refactor@907 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-30 16:56:53 +00:00
Stefano Mazzocchi
aa4de48f95 some renaming, moving tests into main 
git-svn-id: http://google-refine.googlecode.com/svn/branches/split-refactor@906 7d457c2a-affb-35e4-300a-418c747d4874
 2010-05-30 16:55:53 +00:00