Commit Graph

5270 Commits

Author SHA1 Message Date
morrme
66aeaa4409
Remove incorrect "dates" from guess data type label - fixes #2883
Fixes #2883.
2020-07-06 19:55:23 -04:00
Tom Morris
0e832e2d7c
Merge pull request #2889 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-site-plugin-3.9.1
Bump maven-site-plugin from 3.3 to 3.9.1
2020-07-06 16:34:32 -04:00
Tom Morris
f9356dfbd7
Merge pull request #2888 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-shade-plugin-3.2.4
Bump maven-shade-plugin from 3.2.1 to 3.2.4
2020-07-06 16:32:46 -04:00
Tom Morris
f83c6f4376
Adjust Travis build environments - fixes #2861 (#2866)
* Adjust Travis build environments - fixes #2861

Fixes #2861
- Only builds one each of JDK 11-14
- Fixes all validator warnings
- Switches default build environment to bionic
- Uses trusty for an Oracle JDK 8 build
- Adds OS X build
- Adds JDK 13 & 14 builds
- Adds placeholder for JDK 16 builds
  (but Jacoco doesn't currently support it,
   so commented out)
- Reorder build jobs so that most informative ones run first
- Split before_install into before_install and
  before_script sections

* Drop redundant JDK 13 build

* Swap OS X to JDK 14 instead of JDK 13

This doesn't have anything to do with JDK or OS X versions,
but instead the Travis CI build images. A bug in the homebrew
support was only fixed in recent images, so we need to use
an xcode11 build which implies macOS 10.14 or 10.5 and
JDK 14 or 14.0.1.
2020-07-06 20:31:54 +02:00
Ekta Mishra
29f6885ccc
MockConstraintFetcher cleanup part-I (#2882)
* MockConstraintFetcher cleanup part-I

Removed its use from different test files

* Removed MockConstraintFetcher's use from various test files
2020-07-06 18:27:16 +02:00
dependabot-preview[bot]
4ddca58f97
Bump maven-site-plugin from 3.3 to 3.9.1
Bumps [maven-site-plugin](https://github.com/apache/maven-site-plugin) from 3.3 to 3.9.1.
- [Release notes](https://github.com/apache/maven-site-plugin/releases)
- [Commits](https://github.com/apache/maven-site-plugin/compare/maven-site-plugin-3.3...maven-site-plugin-3.9.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:35:49 +00:00
dependabot-preview[bot]
0db676ed3f
Bump maven-shade-plugin from 3.2.1 to 3.2.4
Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.1 to 3.2.4.
- [Release notes](https://github.com/apache/maven-shade-plugin/releases)
- [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.1...maven-shade-plugin-3.2.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:35:16 +00:00
dependabot-preview[bot]
869d889567
Bump maven-deploy-plugin from 2.8.1 to 2.8.2
Bumps maven-deploy-plugin from 2.8.1 to 2.8.2.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:34:45 +00:00
Antonin Delpeuch
ce0126b717
Merge pull request #2884 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-06 07:04:55 +02:00
Ekta Mishra
731692df67
Update MultivalueScrutinizer class (#2885)
Removed inner class as for constraint status and exceptions we now have Constraint class.
2020-07-06 07:03:39 +02:00
Hosted Weblate
f421cfd76f
Merge branch 'origin/master' into Weblate. 2020-07-06 03:42:12 +02:00
Tijs De Schacht
dfcd5a0f25
Translated using Weblate (Dutch)
Currently translated at 54.4% (410 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/nl/
2020-07-06 03:42:08 +02:00
Isao Matsunami
798818400f
Translated using Weblate (Japanese)
Currently translated at 100.0% (193 of 193 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-06 03:42:07 +02:00
Mohamed El ouard Baouche
a6a5935585
Translated using Weblate (Arabic)
Currently translated at 5.8% (44 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ar/
2020-07-06 03:41:49 +02:00
Isao Matsunami
948d1acae1
Translated using Weblate (Japanese)
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-07-06 03:41:49 +02:00
Tom Morris
8a6171432d
Translated using Weblate (French)
Currently translated at 99.0% (746 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-07-06 03:41:48 +02:00
Ekta Mishra
d871a2e151
Implemented RestrictedPosition Scrutinizer tests using mocks (#2868)
* Implemented RestrictedPosition Scrutinizer tests using mocks

Added RestrictedPositionConstraint class and updated test cases using mocks

* Tests updated & working fine
2020-07-04 14:25:00 +02:00
Ekta Mishra
a0a164bb8b
Implemented ResctrictedValue Scrutinizer tests using mocks (#2877)
* Implemented RestrictedValueScrutinizer tests using mocks

Added AllowedValueConstraint & DisallowedValueConstraint classes & updated the test cases too.

* Tests updated
2020-07-04 14:23:52 +02:00
Tom Morris
3717111db8
Fix Open Office Spreadsheet (ODS) dates (#2843)
* Truncate any completely empty columns on the right

Fixes #565
The current versions of Open Office create default spreadsheets
with over 1000 empty columns. Keep track of the rightmost
non-empty column when importing and truncate everything else.

Also adds a basic ODS import test.

* Fix dates in ODS spreadsheets

Fixes #2224
2020-07-04 08:42:33 +02:00
Antonin Delpeuch
952447461f
Fix wikidata logout when credentials have expired. Fixes #2873 (#2878) 2020-07-04 08:38:17 +02:00
Antonin Delpeuch
f4692de9e1 Increase maximum wait for testInvalidUrl, follow-up for #2876 #2875 2020-07-03 21:48:43 +02:00
Tom Morris
df8d092132
Micro benchmark harness & ToNumber optimizations (#2859)
* Performance optimized version of ToNumber

Approximately 5x faster for floats (data dependent)
and about the same speed for integers.

- Instead of blindly trying to parse as Long, do a quick check
  for obvious problems (e.g. decimal point).
- Don't trim. It's already done by called methods.
- Use valueOf() instead of parse() to avoid object creation

* Add Java Microbenchmark Harness

The shaded JAR is missing the OpenRefine classes, for a reason
that I haven't figured out, so requires openrefine-main.jar at runtime.

* Remove old implementations of ToNumber

* Remove unneeded dependencies from main project

* Clean up and reformat
2020-07-03 21:42:44 +02:00
Tom Morris
a88aeca304
Merge pull request #2854 from OpenRefine/dependabot/maven/com.google.apis-google-api-services-sheets-v4-rev20200616-1.30.9
Bump google-api-services-sheets from v4-rev20200508-1.30.9 to v4-rev20200616-1.30.9
2020-07-03 15:27:21 -04:00
Tom Morris
5d6af9cb6c
Merge pull request #2865 from tfmorris/2863-tree-column-ordering
Remove shortest-column-name ordering - fixes #2863
2020-07-03 15:23:36 -04:00
Tom Morris
f5786afa35
Increase test timeout - fixes #2875 (#2876) 2020-07-03 21:20:01 +02:00
Thad Guidry
49fd21759c
remove English sentence from French translation (#2871) 2020-07-03 16:12:43 +02:00
Tom Morris
de2c2aa778
Correct mimetype for Google Drive project exports (#2829)
Fixes #2797. Changes mimetype from zip to gzip
and adds .tar.gz extension to the name.
2020-07-03 14:24:25 +02:00
Tom Morris
139019f6e3
Internationalize clipboard default project name (#2814)
Fixes #2776
2020-07-03 14:22:44 +02:00
Ekta Mishra
c68047a614
Implemented QuantityScrutinizer tests using Mocks (#2862)
* Implemented QunatityScrutinizer tests using Mockito

Updated test cases and added AllowedUnitsConstraint class

* Test cases updated
2020-07-03 14:14:32 +02:00
Ekta Mishra
9edb1e514d
Implemented Difference-within-range Scrutinizer tests using mocks (#2864)
Updated all test cases and added DifferenceWithinRangeConstraint class.
2020-07-03 14:13:31 +02:00
Tom Morris
a4b7a00c70
Merge pull request #2867 from chetan-v/JsonFix
Fixed the guessing of JSON for .txt(2820)
2020-07-03 02:55:50 -04:00
chetan
3932b23eb6 Fixed the guessing of JSON for .txt(2820) 2020-07-03 10:46:07 +05:30
Tom Morris
d3db73aa67 Remove shortest-column-name ordering
Refs #2863
The tree importer sorts columns/column groups by how populated
they are, which is of arguable utility, but the tie-breaker
of ordering by shortest column name is completely silly.

This change removes that and, in conjunction with a stable sort
algorithm, will preserve the original order of the columns.
2020-07-02 16:12:55 -04:00
Tom Morris
28a9f68236
Unit test improvements (#2856)
* Fix two deprecated methods usages

* Test ToNumber conversions

* Test behavior of all functions when passed 0 or 8 arguments

There are 16 which fail currently on 0 args (return null or
False instead of EvalError), but have been whitelisted until
we can verify whether it's safe to change them without introducing
compatibility issues.

There are 19 which fail to return an error on too many (ie 8) args.
2020-07-02 20:29:21 +02:00
Ekta Mishra
cd0ed11dad
Implemented Format Scrutinizer tests using Mockito (#2849)
* Implemented Format Scrutinizer tests using Mockito

Updated implementation of the scrutinzer & tests

* Testcases updated in FormatScrutinizerTest
2020-07-02 16:28:56 +02:00
Ekta Mishra
9dfb9114c4
Implemented QualifierComaptibilty Scrutinizer tests using Mockito (#2860)
Updated test cases & added AlLowedQualifierConstraint and MandatoryQualifierConstraint classes.
2020-07-02 14:22:50 +02:00
Ekta Mishra
67bc8581ce
Implemented InverseScrutinizer tests using Mocks (#2855)
* Implemented InverseScrutinizer tests using Mocks

updated testcases and added InverseConstraint Class

* Test cases updated & working fine
2020-07-01 20:49:15 +02:00
Tom Morris
2d1d740b44
Merge pull request #2853 from OpenRefine/dependabot/maven/com.google.http-client-google-http-client-jackson2-1.36.0
Bump google-http-client-jackson2 from 1.35.0 to 1.36.0
2020-07-01 08:58:19 -04:00
dependabot-preview[bot]
cd0d4bdda9
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200508-1.30.9 to v4-rev20200616-1.30.9.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-01 08:26:36 +00:00
dependabot-preview[bot]
b9dedc4438
Bump google-http-client-jackson2 from 1.35.0 to 1.36.0
Bumps [google-http-client-jackson2](https://github.com/googleapis/google-http-java-client) from 1.35.0 to 1.36.0.
- [Release notes](https://github.com/googleapis/google-http-java-client/releases)
- [Changelog](https://github.com/googleapis/google-http-java-client/blob/master/CHANGELOG.md)
- [Commits](https://github.com/googleapis/google-http-java-client/compare/v1.35.0...v1.36.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-01 08:26:18 +00:00
Thad Guidry
b31d2457b0
Add Chan Zuckerberg Initiaive to our backers file (#2852) 2020-07-01 08:08:11 +02:00
dependabot[bot]
3ec20eecb6
Bump xstream from 1.4.9 to 1.4.10-java7 in /packaging
Bumps [xstream](https://github.com/x-stream/xstream) from 1.4.9 to 1.4.10-java7.
- [Release notes](https://github.com/x-stream/xstream/releases)
- [Commits](https://github.com/x-stream/xstream/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2020-06-30 23:27:53 +00:00
Ekta Mishra
cef2e84e7f
Implemented EntityTypeScrutinizer tests usings mocks (#2839)
Updates all the testcases in EntityTypeScrutinizerTest
2020-06-30 22:59:43 +02:00
Tom Morris
54291ef441
Use Apache IO Commons IOUtils instead of homerolled (#2845)
Probably should remove the funky Gzip support with the
overloaded use of the encoding parameter, but this is
a start.
2020-06-30 13:49:47 +02:00
Chetan Verma
e2a2dd2a4e
Fix misstatement about supported formats in import project screen (#2841)
Closes #2753.
2020-06-30 08:25:15 +02:00
Tom Morris
b64cbfea4f
Fix i18n. Fixes #2805 (#2847)
Fix database extensions exporter which is corrupting the dictionary
name with the value of the language.
2020-06-30 08:22:12 +02:00
Tom Morris
0f3a6006f3
Add Excel95 import test and improve other importer tests (#2844)
No issue.
- we don't support Excel95, but make sure that it generates an exception
- move the test data file into the appropriate directory
- for any normal test, consider exceptions a failure
2020-06-30 08:20:56 +02:00
Tom Morris
421974cc3d
Truncate any completely empty columns on the right (#2842)
Fixes #565
The current versions of Open Office create default spreadsheets
with over 1000 empty columns. Keep track of the rightmost
non-empty column when importing and truncate everything else.

Also adds a basic ODS import test.
2020-06-30 08:19:00 +02:00
Ekta Mishra
bc672047f6
Implemented DistinctValueScrutinizer tests using mockito (#2833)
* Implemented DistinctValueScrutinizer tests using mcokito

Added inner class to the scrutinizer and updated the tests using mocks.

* Tests updated-testNoIssue added

* all tests updated & working fine
2020-06-29 16:00:37 +02:00
Ekta Mishra
46c510b5e2
Implemented SingleValue Scrutinizer tests using mocks (#2818)
* Implemented SingleValue Scrutinizer tests using mocks

Updated test class & added inner class to the scrutinizer

* tests updated

* Updated SingleValueConstraint class
2020-06-29 15:59:53 +02:00