Commit Graph

5348 Commits

Author SHA1 Message Date
Tom Morris
50bb76e53a
Merge pull request #2929 from OpenRefine/dependabot/maven/com.google.apis-google-api-services-sheets-v4-rev20200616-1.30.10
Bump google-api-services-sheets from v4-rev20200616-1.30.9 to v4-rev20200616-1.30.10
2020-07-13 12:49:36 -04:00
Tom Morris
3958c57a17
Merge pull request #2928 from OpenRefine/dependabot/maven/com.squareup.okhttp3-mockwebserver-4.8.0
Bump mockwebserver from 4.7.2 to 4.8.0
2020-07-13 12:38:21 -04:00
Tom Morris
74d959018f
Merge pull request #2927 from OpenRefine/dependabot/maven/mysql-mysql-connector-java-8.0.21
Bump mysql-connector-java from 8.0.20 to 8.0.21
2020-07-13 12:34:33 -04:00
dependabot-preview[bot]
498325ded9
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200616-1.30.9 to v4-rev20200616-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-13 08:39:42 +00:00
dependabot-preview[bot]
396efc0d1b
Bump mockwebserver from 4.7.2 to 4.8.0
Bumps [mockwebserver](https://github.com/square/okhttp) from 4.7.2 to 4.8.0.
- [Release notes](https://github.com/square/okhttp/releases)
- [Changelog](https://github.com/square/okhttp/blob/master/CHANGELOG.md)
- [Commits](https://github.com/square/okhttp/compare/parent-4.7.2...parent-4.8.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-13 08:39:23 +00:00
dependabot-preview[bot]
906e9d1279
Bump mysql-connector-java from 8.0.20 to 8.0.21
Bumps [mysql-connector-java](https://github.com/mysql/mysql-connector-j) from 8.0.20 to 8.0.21.
- [Release notes](https://github.com/mysql/mysql-connector-j/releases)
- [Changelog](https://github.com/mysql/mysql-connector-j/blob/release/8.0/CHANGES)
- [Commits](https://github.com/mysql/mysql-connector-j/compare/8.0.20...8.0.21)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-13 08:39:00 +00:00
dependabot-preview[bot]
940c508584
Bump google-api-services-drive
Bumps google-api-services-drive from v3-rev20200618-1.30.9 to v3-rev20200618-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-13 08:38:36 +00:00
Tom Morris
5717d17b50
Merge pull request #2921 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-12 14:32:42 -04:00
Hosted Weblate
69dc478450
Merge branch 'origin/master' into Weblate. 2020-07-12 13:41:53 +02:00
Susanna Ånäs
fbd28f960c
Translated using Weblate (Finnish)
Currently translated at 23.3% (45 of 193 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/fi/
2020-07-12 13:41:51 +02:00
Allan Nordhøy
98b64b7a01
Translated using Weblate (Norwegian Bokmål)
Currently translated at 51.6% (389 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/nb_NO/
2020-07-12 13:41:50 +02:00
Rafael Fontenelle
f329f5085b
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/pt_BR/
2020-07-12 13:41:49 +02:00
Tom Morris
d209039a6e
Merge pull request #2918 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-11 14:02:56 -04:00
Hosted Weblate
6c69525545
Merge branch 'origin/master' into Weblate. 2020-07-11 12:55:18 +02:00
Allan Nordhøy
128a3089ed
Translated using Weblate (Norwegian Bokmål)
Currently translated at 51.1% (385 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/nb_NO/
2020-07-11 12:55:18 +02:00
Rafael Fontenelle
b0177e6e33
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/pt_BR/
2020-07-11 12:55:17 +02:00
Isao Matsunami
1a7ae77431
Translated using Weblate (Japanese)
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-07-11 12:55:17 +02:00
Susanna Ånäs
16695e54b5
Added translation using Weblate (Finnish) 2020-07-11 12:55:13 +02:00
Tom Morris
306b541c69
Fix Excel date import - Fixes #1908 (#2909)
* Add utility functions to check/convert dates

* Add date tests and refactor to DRY up

* Fix date import - fixes #1908

Change from java.util.Date to OpenRefine 3.0+'s OffsetDateTime
Fixes #1908

* Centralize date conversion

* Moving utility methods to ParsingUtilities

* Fix tests
2020-07-09 23:13:44 +02:00
Urvashi Gupta
a0f2d11255
addStandardServiceOnEnter (#2914) 2020-07-09 22:57:24 +02:00
dependabot-preview[bot]
a0a889688b
Bump google-api-services-drive (#2911)
Bumps google-api-services-drive from v3-rev20200609-1.30.9 to v3-rev20200618-1.30.9.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-09 10:29:00 +02:00
dependabot-preview[bot]
380942d53f
Bump httpmime from 4.5.2 to 4.5.12 (#2904)
Bumps httpmime from 4.5.2 to 4.5.12.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-08 10:29:09 +02:00
Tom Morris
f23e3096f3
Tweak CI build settings - refs #2861 (#2894)
* Sleep to wait for database servers on Mac - refs #2861

* Tweak JDK build settings (again) - refs #2861

- Use default JDK on platforms where possible
- Make problematic builds (requiring Java installs) optional
2020-07-07 21:41:55 +02:00
Tom Morris
0562638ffa
Use standard text normalization - fixes #2898 (#2900)
* Use standard text normalization - fixes #2898

Fixes #2898. Fixes #409. Refs #650

Replaces homegrown ISO Latin-1 only character subsitition
with standard Java Normalize to NFD, followed by diacritic
removal and a few custom character expansions/replacements.

* Fix Mac build

* Improve compatibility with previous code

One intentional change is folding O with stroke to
oe instead of o.

- Use more powerful NFKD instead of NFD
- strip punctuation after decomposition since it can generate
  new punctuation
- Add compatibility test for old asciify() method
- Add some graphically similar characters to substitution table

* Add oe character/ligature & more long S forms

* More tests for ligatures and Latin Extended

* Add Latin-1 Supplement tests
2020-07-07 21:35:41 +02:00
allanaaa
ab2ec8d28f
Adding mostly-empty docs (#2892)
Skeleton and initial content for documentation
2020-07-07 15:16:27 -04:00
Antonin Delpeuch
48b5918408 Rename nightly releases to snapshot releases 2020-07-07 17:49:59 +02:00
Urvashi Gupta
f62f63706c
Report HTTP error codes to the user when creating a project from a URL (#2870)
* HTTP Error

* urlImportingTestCompleted
2020-07-07 11:58:47 +02:00
Tom Morris
e61d50a1aa
Fix NGramFingerprintKeyer to ignore accents - fixes #1161 (#2899)
Fixes #1161
This change parallels what was done in #1257 1da3c00 to fix
the FingerprintKeyer and moves the diacritic removal before
the deduping. Includes a test.
2020-07-07 09:02:49 +02:00
morrme
66aeaa4409
Remove incorrect "dates" from guess data type label - fixes #2883
Fixes #2883.
2020-07-06 19:55:23 -04:00
Tom Morris
0e832e2d7c
Merge pull request #2889 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-site-plugin-3.9.1
Bump maven-site-plugin from 3.3 to 3.9.1
2020-07-06 16:34:32 -04:00
Tom Morris
f9356dfbd7
Merge pull request #2888 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-shade-plugin-3.2.4
Bump maven-shade-plugin from 3.2.1 to 3.2.4
2020-07-06 16:32:46 -04:00
Tom Morris
f83c6f4376
Adjust Travis build environments - fixes #2861 (#2866)
* Adjust Travis build environments - fixes #2861

Fixes #2861
- Only builds one each of JDK 11-14
- Fixes all validator warnings
- Switches default build environment to bionic
- Uses trusty for an Oracle JDK 8 build
- Adds OS X build
- Adds JDK 13 & 14 builds
- Adds placeholder for JDK 16 builds
  (but Jacoco doesn't currently support it,
   so commented out)
- Reorder build jobs so that most informative ones run first
- Split before_install into before_install and
  before_script sections

* Drop redundant JDK 13 build

* Swap OS X to JDK 14 instead of JDK 13

This doesn't have anything to do with JDK or OS X versions,
but instead the Travis CI build images. A bug in the homebrew
support was only fixed in recent images, so we need to use
an xcode11 build which implies macOS 10.14 or 10.5 and
JDK 14 or 14.0.1.
2020-07-06 20:31:54 +02:00
Ekta Mishra
29f6885ccc
MockConstraintFetcher cleanup part-I (#2882)
* MockConstraintFetcher cleanup part-I

Removed its use from different test files

* Removed MockConstraintFetcher's use from various test files
2020-07-06 18:27:16 +02:00
dependabot-preview[bot]
4ddca58f97
Bump maven-site-plugin from 3.3 to 3.9.1
Bumps [maven-site-plugin](https://github.com/apache/maven-site-plugin) from 3.3 to 3.9.1.
- [Release notes](https://github.com/apache/maven-site-plugin/releases)
- [Commits](https://github.com/apache/maven-site-plugin/compare/maven-site-plugin-3.3...maven-site-plugin-3.9.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:35:49 +00:00
dependabot-preview[bot]
0db676ed3f
Bump maven-shade-plugin from 3.2.1 to 3.2.4
Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.1 to 3.2.4.
- [Release notes](https://github.com/apache/maven-shade-plugin/releases)
- [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.1...maven-shade-plugin-3.2.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:35:16 +00:00
dependabot-preview[bot]
869d889567
Bump maven-deploy-plugin from 2.8.1 to 2.8.2
Bumps maven-deploy-plugin from 2.8.1 to 2.8.2.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-06 08:34:45 +00:00
Antonin Delpeuch
ce0126b717
Merge pull request #2884 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-06 07:04:55 +02:00
Ekta Mishra
731692df67
Update MultivalueScrutinizer class (#2885)
Removed inner class as for constraint status and exceptions we now have Constraint class.
2020-07-06 07:03:39 +02:00
Hosted Weblate
f421cfd76f
Merge branch 'origin/master' into Weblate. 2020-07-06 03:42:12 +02:00
Tijs De Schacht
dfcd5a0f25
Translated using Weblate (Dutch)
Currently translated at 54.4% (410 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/nl/
2020-07-06 03:42:08 +02:00
Isao Matsunami
798818400f
Translated using Weblate (Japanese)
Currently translated at 100.0% (193 of 193 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-06 03:42:07 +02:00
Mohamed El ouard Baouche
a6a5935585
Translated using Weblate (Arabic)
Currently translated at 5.8% (44 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ar/
2020-07-06 03:41:49 +02:00
Isao Matsunami
948d1acae1
Translated using Weblate (Japanese)
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-07-06 03:41:49 +02:00
Tom Morris
8a6171432d
Translated using Weblate (French)
Currently translated at 99.0% (746 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-07-06 03:41:48 +02:00
Ekta Mishra
d871a2e151
Implemented RestrictedPosition Scrutinizer tests using mocks (#2868)
* Implemented RestrictedPosition Scrutinizer tests using mocks

Added RestrictedPositionConstraint class and updated test cases using mocks

* Tests updated & working fine
2020-07-04 14:25:00 +02:00
Ekta Mishra
a0a164bb8b
Implemented ResctrictedValue Scrutinizer tests using mocks (#2877)
* Implemented RestrictedValueScrutinizer tests using mocks

Added AllowedValueConstraint & DisallowedValueConstraint classes & updated the test cases too.

* Tests updated
2020-07-04 14:23:52 +02:00
Tom Morris
3717111db8
Fix Open Office Spreadsheet (ODS) dates (#2843)
* Truncate any completely empty columns on the right

Fixes #565
The current versions of Open Office create default spreadsheets
with over 1000 empty columns. Keep track of the rightmost
non-empty column when importing and truncate everything else.

Also adds a basic ODS import test.

* Fix dates in ODS spreadsheets

Fixes #2224
2020-07-04 08:42:33 +02:00
Antonin Delpeuch
952447461f
Fix wikidata logout when credentials have expired. Fixes #2873 (#2878) 2020-07-04 08:38:17 +02:00
Antonin Delpeuch
f4692de9e1 Increase maximum wait for testInvalidUrl, follow-up for #2876 #2875 2020-07-03 21:48:43 +02:00
Tom Morris
df8d092132
Micro benchmark harness & ToNumber optimizations (#2859)
* Performance optimized version of ToNumber

Approximately 5x faster for floats (data dependent)
and about the same speed for integers.

- Instead of blindly trying to parse as Long, do a quick check
  for obvious problems (e.g. decimal point).
- Don't trim. It's already done by called methods.
- Use valueOf() instead of parse() to avoid object creation

* Add Java Microbenchmark Harness

The shaded JAR is missing the OpenRefine classes, for a reason
that I haven't figured out, so requires openrefine-main.jar at runtime.

* Remove old implementations of ToNumber

* Remove unneeded dependencies from main project

* Clean up and reformat
2020-07-03 21:42:44 +02:00