Commit Graph

360 Commits

Author SHA1 Message Date
Antonin Delpeuch
4013c0b8c5 Add (de)serialization tests to existing operation test classes 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
dbff74650d Cleanup imports 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
8896636017 Add deserialization tests for operations 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
2edf83bdf7 Start serialization tests for operations 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
fac8936411 Serialization tests for model classes 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
c6d2b003b1 Start serialization tests 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
d9bff34411 Add testing utilites for JSON serialization 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
2b57350adf Cleanup StandardReconConfigTests 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
b12b29b393 Rename ReconTests 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
3ea1822773 Remove duplicate test class introduced by Jacky 2018-09-01 12:40:23 +02:00
Antonin Delpeuch
f41678e087 Move tests to their relevant package. 2018-09-01 12:40:22 +02:00
Antonin Delpeuch
94b6b1f945
Merge pull request #1692 from OpenRefine/issue1689
Fix exception when sorting the empty string numerically
2018-08-30 08:09:20 +02:00
Jacky
21712db6e0
Merge pull request #1698 from OpenRefine/issue1681
Fix unescape GREL function in javascript mode.
2018-08-29 21:24:52 -04:00
Antonin Delpeuch
d01f916101 Remove test assertion that can fail depending on the locale 2018-08-08 11:51:10 +01:00
TonyO
c1af2f1179 fixed issue 1684 with decimal values 2018-08-07 11:25:49 +01:00
Antonin Delpeuch
09bb2903f2 Add test for sorting exception, issue #1689 2018-08-07 11:24:22 +01:00
Antonin Delpeuch
18c489855b Test for clickable URLs in HTML exporter 2018-08-07 11:23:49 +01:00
Antonin Delpeuch
87b6aadbad Fix unescape GREL function in javascript mode. Closes #1681. 2018-08-07 11:23:14 +01:00
Owen Stephens
64ed24e082 Small amount of tidying up code 2018-06-23 00:28:45 +01:00
Owen Stephens
420c9bd1a5 Add tests for grouping of values in text facets 2018-06-23 00:18:00 +01:00
Owen Stephens
d309af7f1a Update string conversion tests to reflect new behaviour of stringUtils 2018-06-23 00:18:00 +01:00
Owen Stephens
a159e76a62 Added test for reconstructing mass edits on dates 2018-06-23 00:18:00 +01:00
Antonin Delpeuch
9f7d5b8786
Merge pull request #1638 from OpenRefine/issue1637
Do not discard the recon space when marking cells as new items.
2018-06-22 12:23:15 +02:00
Jacky
f3dbc18214
Merge pull request #1642 from ostephens/mass-edit-fix
Mass edit fix for blanks, numbers and booleans
2018-06-20 22:00:34 -04:00
Antonin Delpeuch
448a969cb1
Merge pull request #1656 from OpenRefine/issue/1648
fix the template load issue and package typo
2018-06-17 09:23:52 +01:00
Jacky
5a6c9d72f0 fix the template load issue and package typo 2018-06-16 15:43:29 -04:00
Owen Stephens
5ecb1afe13 Updated toString test to new behaviour for null.toString() 2018-06-14 12:36:01 +01:00
Owen Stephens
92e456d2ff More tidying of tests 2018-06-08 20:09:29 +01:00
Owen Stephens
2e9708c16d Tidying up test file 2018-06-08 20:03:38 +01:00
Thad Guidry
057b64fcc2
Increasing to 10 second sleep because of Travis
In Travis these tests are occasionally failing https://travis-ci.org/OpenRefine/OpenRefine/jobs/3896227 which is probably due to the CPU demand spike placed with createProcess.
2018-06-08 06:26:08 -05:00
Owen Stephens
97ea151134 Removed date tests 2018-06-08 09:40:50 +01:00
Owen Stephens
256a81e308 Initial MassOperationTests
Corrected and expanded tests for ReconstructEdit

Improve tests
2018-06-08 09:40:35 +01:00
Antonin Delpeuch
a63151bbcc Do not discard the recon space when marking cells as new items.
Fixes #1637
2018-06-04 12:16:33 +01:00
Jacky
345be88de8 fix UT testHttpHeaders 2018-05-29 22:43:17 -04:00
Jacky
476ece5a67
Merge pull request #1617 from OpenRefine/issue1600
Remove DataExtensions from change serialization
2018-05-24 15:00:34 -04:00
Antonin Delpeuch
d8f081b977 Remove DataExtensions from change serialization 2018-05-22 23:09:51 +02:00
Antonin Delpeuch
b01177c0d9
Merge pull request #1597 from OpenRefine/issue/1588-2
unify the internal date type
2018-05-18 22:03:14 +02:00
Owen Stephens
19a134d4ae Corrected spelling of Calender to Calendar 2018-05-17 10:01:04 +01:00
Jacky
52a2ec05e1
Merge pull request #1598 from ostephens/no-field-null
No field null
2018-05-16 16:01:05 -04:00
Jacky
a103e1fb90 merge the translation 2018-05-16 15:40:05 -04:00
Jacky
147616f41c add more UT for tostring 2018-05-14 22:01:06 -04:00
Jacky
5adb3e455c fix the date format bug 2018-05-14 21:25:47 -04:00
Jacky
8fe27eff58 use yyyy-mm-dd formatter when converting date to string 2018-05-12 20:19:55 -04:00
Owen Stephens
1e2c0c0a04 Added tests for getting non-existent fields and changed test for parser eval error 2018-05-10 16:37:11 -04:00
Jacky
cf5c884ca0 fix minor issues 2018-05-09 23:15:51 -04:00
Jacky
c1132aa58a add nano support 2018-05-09 22:57:10 -04:00
Jacky
a53d0970ea clean up the UT 2018-05-08 22:13:47 -04:00
Jacky
f58d963dbd Merge branch 'issue/1588' of https://github.com/OpenRefine/OpenRefine into issue/1588-2 2018-05-07 17:26:02 -04:00
Antonin Delpeuch
8c2c4546d4
Merge pull request #1568 from OpenRefine/fix-mark-new-items
Fix identifier space for cells reconciled to New
2018-05-06 08:45:25 +02:00
Antonin Delpeuch
b8472cf6b8
Merge pull request #1573 from OpenRefine/issue472
Fix FillDown and BlankDown operations on records.
2018-05-06 08:45:01 +02:00
Jacky
19f98b7ea2 java 8 time API migrate 2018-05-05 11:41:40 -04:00
Jacky
6787ab881e implement nano date part UT 2018-05-02 22:02:57 -04:00
Jacky
34c46111ea undo the incidently push to master 2018-05-02 21:40:23 -04:00
Jacky
47d6f79e35 implement nano data part 2018-05-02 21:23:17 -04:00
TonyO
0eb7b4082f added null and default value options 2018-04-26 20:50:46 -05:00
Antonin Delpeuch
1daa4d1312 Fix date parsing.
Closes #1564.
2018-04-22 10:49:22 +02:00
Antonin Delpeuch
53c12ca2df Fix FillDown and BlankDown operations on records.
Closes #472.
2018-04-21 22:11:54 +02:00
Antonin Delpeuch
716a1606b4
Merge pull request #1558 from gobertm/master
Contains function supports now regex as second argument
2018-04-19 10:46:15 +02:00
Maxime Gobert
304f65045b Contains function tests 2018-04-18 12:17:26 +02:00
Antonin Delpeuch
2b237433f9 Fix identifier space for cells reconciled to New 2018-04-15 01:20:25 +02:00
Jacky
de42f4d85a migrate from JRDF to JENA library 2018-04-08 22:38:23 -04:00
TonyO
3834c27c8f SQL Export Code Commit 2018-03-24 22:41:07 -05:00
Antonin Delpeuch
c536584991
Merge pull request #1536 from ostephens/coalesce
Add coalesce function
2018-03-18 13:06:52 +00:00
Owen Stephens
6e02baaa8a Coalesce must have at least two arguments 2018-03-17 16:21:05 +00:00
Owen Stephens
998eefc869 Make codacy happier 2018-03-14 23:59:13 +00:00
Owen Stephens
74a6782ca0
Merge pull request #1524 from OpenRefine/issue/1508
support project backward compatibility to old version of OpenRefine
2018-03-07 09:36:13 +00:00
Owen Stephens
01b96a0155 Add coalesce function 2018-03-07 09:10:11 +00:00
Jacky
23ec54b78f support project backward compatibility to old version of OpenRefine 2018-02-28 20:16:29 -05:00
Jacky
54553da733
Merge pull request #1518 from OpenRefine/issue/1509
Fix issue #1509
2018-02-28 19:42:07 -05:00
Jacky
ab26bf4a27 fix issue #1517, limit is writen as int and should have been read as int 2018-02-25 17:22:34 -05:00
Jacky
73c5c640cc tidy up import 2018-02-25 15:43:37 -05:00
Jacky
036866837a add UT for issue #1509 2018-02-25 15:31:59 -05:00
Antonin Delpeuch
3dcb3f1faf
Merge pull request #1515 from ostephens/conversions
Conversions
2018-02-22 09:49:49 +00:00
Owen Stephens
2758633f32 Update toDate and toNumber tests to handle null and empty strings correctly 2018-02-21 23:10:38 +00:00
Owen Stephens
731e1310c6 Stripping stuff we don't need out of the tests 2018-02-19 13:42:45 +00:00
Owen Stephens
d2687cc58a Treat null and empty string as different values when doing transforms 2018-02-18 23:19:28 +00:00
Joanne
e0d87c3d94 Update tests to match updated range function 2018-02-16 15:13:37 +00:00
Joanne
01d8e458f4 Update tests to match updated range function 2018-02-16 15:13:37 +00:00
Joanne
e2793daf54 Update tests for improved range function 2018-02-16 15:13:37 +00:00
Joanne
d95f5985b9 Add tests for range function 2018-02-16 15:13:37 +00:00
Owen Stephens
0d04a25cf1
Merge pull request #1450 from OpenRefine/issue1448
Make Wikitext importer more robust
2018-02-07 17:41:23 +00:00
Antonin Delpeuch
f5ff5565ff Cleanup for Codacy and better parsing of quote character 2018-02-03 16:37:15 +00:00
Antonin Delpeuch
a89bbcbbe2 Add missing method removed during merge 2018-02-03 14:29:24 +00:00
Thad Guidry
c83dbe3b23
Merge branch 'master' into stundzig-develop/1086-quotecharacter 2018-02-03 07:20:45 -06:00
Antonin Delpeuch
c9395d564c Merge branch 'develop/1086-quotecharacter' of https://github.com/stundzig/OpenRefine into stundzig-develop/1086-quotecharacter 2018-02-03 07:56:30 +00:00
Jacky
c4b0ff6bea data package metadata (#1398)
* fix the appbundle issue #1209

* fix #1162

allow the JRE 9

* fix the package declarations

* remove the _ from the method name

* use the explicit scoping

* remote extra ;

* fix issued from codacy

* fix issued from codacy

* add preferences link to the index page

* handle the empty user metadata

* fix 'last modified' sorting issue #1307

* prevent overflow of the table. issue #1306

* add isoDateParser to sort the date

* prevent overflow of the project index

* remove sorter arrow for action columns

* disable editing the internal metadata

* adjust the width of the table

* change MetaData to Metadata

* change the filed name from rowNumber to rowCount

* put back the incidently deleted gitignore

* add double quote to prevent word splitting

* UI improvement on metadata view and project list view

* remove the date field in metadata

* message notification of the free RAM. Issue #1295

* UI tuning for metadata view

* shorten the ISO date to locale date format

* Added translation using Weblate (Portuguese (Brazil))

* remove the rename link

* Ignore empty language files introduced by Weblate

* Add UI for Invert text filter

* Backend support for Inverting Text search facets

* Fix reset on text search facet

* More succinct return statements

* add tests for SetProjectMetadataCommand

* Tidying up for Codacy

* Added Tests for TextSearchFilter

* Corrections for Codacy

* More code tidy up

* let the browser auto fit the table cell when resizing/zooming

* fix import multiple excel with mulitple sheets issue #1328

* check if the project has the userMetadata

* fix the unit test
support multi files with multi tables for open office

* prevent the same key for user metadata

* replace _ with variable for exception

* fix the no-undef issue

* to adjust the width of transform dialog. issue #1332

* fix the row count refresh issue

* extract method

* move the log message

* cosmatic changes for codacy

* fix typo

* bump to version 2.8

* .gitignore is now working

* preview stage won't have the metadata populated, so protect NPE

* Update README.md

No more direct link to the last version tag, which will avoid having to think of updating the readme

* refacotring the ProjectMetadata class

* introduce the IMetadata interface

* create submodule of dataschema

* add back

* setup lib for dataschema; upgrade the apache lang to lang3

* replace escape* functions from apache lang3

* replace the ProjectMetadata with IMetadata interface

* add missing jars

* set the IMetadata a field of Project

* remove PreferenceStore out of Project model

* fix test SetProjectMetadataCommandTests by casting

* introdcue the AbstractMetadata

* introdcue the AbstractMetadata

* reorganize the metadata package

* allow have mulitiple metadata for a project

* support for mulitple metadata format

* remove jdk7 since 'table schema' java implmentation only support jdk8+

* set execute permission for script

* fix the Unit Test after Metadata refactoring

* restore the apache lang2.5 since jetty 6.1.22 depend on it

* add commons lang 2.5 jar

* git submodule add  https://github.com/frictionlessdata/datapackage-java

* remove the metadata parameter from the ProjectManager.registerProject method

* remove hashmap _projectsMetadata field from the ProjectManager and FileProjectManager

* init the Project.metadataMap

* fix Unit Test

* restore the ProjectMetaData map to ProjectManager

* put the ProjectMetaDta in place for ProjectManager and Project object

* check null of singleton instead of create a constructor just for test

* load the data package metadata

* importing data package

* importing data package

* encapsulate the Package class into DataPackageMetadata

* user _ to indicate the class fields

* introduce base URL in order to download the data files

* import data package UI and draft backend

* import data package UI

* fix typo

* download the data set pointed from metadata resource

* save and load the data package metadata

* avoid magic string

* package cleanup

* set the java_version to 1.8

* set the min jdk to 1.8

* add the 3rd party src in the build.xml

* skip the file selection page if only 1 DATA file

* add files structure for json editor

* seperate out the metadata file from the retrival file list

* rename the OKF_METADATA to DATAPACKAGE_METADATA

* clean up

* implement GetMetadateCommand class

* display the metadata in json format

* git submodule update --remote --merge

* adjust the setting after pulling from datapackage origin

* fix the failed UT DateExtensionTests.testFetchCounts due to new json jar json-20160810.jar will complain: JSONObject["float"] not a string.

* clean up the weird loop array syntax get complained

* remove the unused constant

* export in data package format

* interface cleanup

* fix UT

* edit the metadata

* add UT for SetMetadataCommand

* fix UT for SetMetadataCommand

* display the data package metadata link on the project index page

* update submodule

* log the exceptions

* Ajv does not work properly, use the back end validation instead

* enable the validation for jsoneditor

* first draft of the data validation

* create a map to hold the constraint and its handler

* rename

* support for minLength and maxLength from spec

* add validate command

* test the opeation instead of validate command

* rename the UT

* format the error message and push to the report

* fix row number

* add resource bundle for validator

* inject the code of the constrains

* make the StrSubstitutor works

* extract the type and format information

* add the customizedFormat to interface to allow format properly

* get rid of magic string

* take care of missing parts of the data package

* implement RequiredConstraint

* patch for number type

* add max/min constraints

* get the constrains directly from field

* implement the PatternConstraint

* suppress warning

* fix the broken UT when expecting 2 digits fraction

* handle the cast and type properly

* fix the missing resource files for data package when run from command line

* use the copy instead of copydir

* add script for appveyor

* update script for appveyor

* do recursive clone

* correct the git url

* fix clone path

* clone folder option does not work

* will put another PR for this. delete for now

* revert the interface method name

* lazy loading the project data

* disable the validate menu for now

* add UT

* assert UTs

* add UT

* fix #1386

* remove import

* test the thread

* Revert "test the thread"

This reverts commit 779214160055afe3ccdcc18c57b0c7c72e87c824.

* fix the URLCachingTest UT

* define the template data package

* tidy up the metadata interface

* check the http response code

* fix the package

* display user friendly message when URL path is not reachable

* populate the data package schema

* Delete hs_err_pid15194.log

* populate data package info

* add username  preference and it will be pulled as the creator of the metadata

* undo the project.updateColumnChange() and start to introduce the fields into the existing core model

* tightly integrate the data package metadata

* tightly integrate the data package metadata for project level

* remove the submodule

* move the edit botton

* clean up build

* load the new property

* load the project metadata

* fix issues from codacy

* remove unused fields and annotation

* check the http response code firstly

* import zipped data package

* allow without keywords

* process the zip data package from url

* merge the tags

* check store firstly

* remove the table schema src

* move the json schema files to schema dir

* add comment

* add comment

* remove git moduels

* add incidently deleted file

* fix typo

* remove SetMetadataCommand

* revert change

* merge from master
2018-02-02 13:24:19 +00:00
Antonin Delpeuch
47f44b06c4 Make Codacy happy 2018-02-02 11:57:20 +00:00
Antonin Delpeuch
1f87a26f4c Include image file names like templates in Wikitable importer 2018-02-02 11:22:15 +00:00
Antonin Delpeuch
843641ca74 Parse headers just like other cells in Wikitable importer
This makes the column names a bit less meaningful,
but this is necessary to handle cases where headers
are inserted anywhere in the table.
2018-02-02 10:48:31 +00:00
Antonin Delpeuch
0426704938 Add helper createProjectWithColumns 2018-01-27 13:58:06 +00:00
Antonin Delpeuch
feae25a5ff Migrate more tests to the createCSVProject helper 2018-01-27 13:45:44 +00:00
Antonin Delpeuch
8f9fc61046 Migrate TextSearchFacetTests.java to use the helpers 2018-01-27 13:45:44 +00:00
Antonin Delpeuch
5278e929a2 Refactor CrossFunctionTests' project creating logic into RefineTest 2018-01-27 13:45:44 +00:00
Owen Stephens
4320ef8c10 Resolved merge conflict 2018-01-16 17:41:56 +00:00
Owen Stephens
5acdb1429c Fix tests - Include new httpHeaders JSON array when creating new ColumnAdditionByFetchingURLsOperation 2018-01-16 17:22:48 +00:00
Jacky
3c25647f05 add find function. issue 1296 2018-01-14 22:04:26 -05:00
Olle Jonsson
76fa942490 Nobel prize winners CSV data typos 2018-01-07 21:28:07 +01:00
Antonin Delpeuch
45ccd0e1c0 Change HTTP error detection in URL fetching test 2018-01-03 22:44:32 +01:00
Antonin Delpeuch
dae4ac1ce4 Simpler way to pass the test when there are 503 errors 2018-01-03 19:03:26 +01:00
Antonin Delpeuch
32b7d2fa74 Looser check for 403 and 503 errors in URL caching test 2018-01-03 18:57:34 +01:00