Commit Graph

5524 Commits

Author SHA1 Message Date
Tom Morris
392a64b25e Refactor tests to hoist common methods into RefineTest
Moves the invoke() method and the associated fields into
the super class and deletes the redundant implementations.
2020-08-23 14:11:49 -04:00
Antonin Delpeuch
9ac54edbba
Migrate reconciliation calls to Apache HTTP client (#2906)
* Migrate reconciliation calls to OkHTTP, for #2903

* Migrate to Apache HTTP Commons

* Migrate data extension to Apache HTTP client

* Deprecate HttpURLConnection in RefineServlet

* Use LaxRedirectStrategy, clean up imports

* Remove read and pool timeouts, only keep the connection timeout

* Adapt mocking of HTTP calls after migration
2020-08-23 14:04:59 +02:00
Tom Morris
259705ad5f
assertEqualAsJson test helper refactor (#3113)
* Refactor test helper

Create a version of the assert that uses the standard parameter
order and deprecate the version that uses inverted order.

* Use consistent Assert class and parameter ordering
2020-08-23 11:04:44 +02:00
Tom Morris
42e840dec6
Merge pull request #3110 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-22 12:17:37 -04:00
Isao Matsunami
9470c47031 Translated using Weblate (Japanese)
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-08-22 18:14:45 +02:00
Lu Liu
7d4e182c75
Extend Wikidata extension to support arbitrary Wikibase instances (#2810)
Closes #1640. All Wikibase-dependent parameters, which were previously hard-coded for Wikidata, are now described in a JSON manifest. The manifest is currently constructed by hand, but, in the future, will hopefully be published by each Wikibase instance at a standard location.

* setup the manifest framework

* add dependency mechanism to scrutinizers & update tests

* add json creators to constraint entities

* adapt the backend (units tests are to be updated)

* remove the call to prepareDependencies() in the constructor

* update code according to review feedback

* update scrutinizers tests

* fix typo & update ConstraintsV1

* log if a scrutinizer is skipped

* update versioning handling in the backend

* correct the order of "actual" and "expected" for assertEquals method

* use regex to check manifest versions

* 1. add wikibase-manager.js, wikibase-dialog.js, etc.
2. move dialog/schema-alignment-dialog.js -> schema-alignment.js
3. remove unused schema-alignment-dialog.html
4. change most mentions of "Wikidata" to "Wikibase"

* support saving cookies for different Wikibases & fix LoginCommandTest

* fix schema related tests

* removed unused WikibaseCredentials

* include MediaWiki API endpoint in the schema

* fetch language codes for different Wikibases

* fix lgtm-bot alerts

* keep a connection map (MediaWiki API endpoint => Connection) in ConnectionManager

* simplify the constraint configurations of the manifest and remove lots of unnecessary code.

* add slash to the end of mediawiki.root

* add manifest schema and use ajv to validate the manifest

* remove JSONP support (Wikibase manifest host should support CORS)

* save manifests on manifest update

* add unit tests for Manifest

* include the exception in logger.error() method to make it easier to debug

* include the message of ManifestException when calling respondError

* test multiple connections

* test no manifest & test invalid manifest

* adapt manage-account-dialog.js to support multiple Wikibase connections

* update instance/subclass of related translations

* beautify import-schema-dialog.html

* use "${lang}" variable in the reconciliation service endpoint of the manifest

* adapt schema-alignment.js after introducing "${lang}" variable in the reconciliation service endpoint

* use WikibaseManager.getSelectedWikibaseApi() in SchemaAlignment._getPropertyType

* replace more mentions of "Wikidata" to "Wikibase"

* use WikibaseManager.getSelectedWikibaseApi() in previewrenderer.js

* support fetching language codes of different Wikibases in the frontend

* skip EditInspector if missing 'property_constraint_pid' in the manifest

* improve unit tests for fetching lang codes

* skip scrutinizers depending on fetcher if 'property_constraint_pid' is missing in the manifest

* make sure the schema alignment panel is set up before rendering

* fix preview bug

* add getters of "instance of" and "subclass of" to the Manifest interface and use them in NewItemScrutinizer

* fix hardcode for Wikidata in WbItemVariable

* rename 'entity_prefix' to 'site_iri' and move it from 'manifest.wikibase.properties' to 'manifest.wikibase'

* include oauth configurations in the manifest & support logging in with owner-only consumer for Wikibases with the OAuth extension

* correct schema fallback logic

* select default wikibase according to the saved schema

* include maxlag in the manifest

* [backend] move maxlag setting from preferences to request parameter

* support setting maxlag when uploading edits

* rename "Manage Wikibase" to "Select Wikibase instance" and localize it

* fix manifest updating bug

* include EditGroups in the manifest

* add the reconciliation service from the manifest to standard services if it's not present yet when adding a new manifest

* update according to review feedback

1. use inherited color variable
2. rename 'gridwroks' to 'openrefine'
3. remove unnecessary 'async: true'
4. add 'format: url' validation to urls to the schema

* rename 'wikibasePrefix' to 'siteIri'
2020-08-22 11:58:56 -04:00
Antonin Delpeuch
68404abe2a
Fix race condition in service manifest fetching (#2872)
* Fix race condition in service manifest fetching. Fixes #2803

* Set own recon endpoint for Wikidata extension
2020-08-21 07:15:02 +02:00
Tom Morris
92ad819e34
Merge pull request #3100 from OpenRefine/dependabot/maven/org.mockito-mockito-core-3.5.2
Bump mockito-core from 3.5.0 to 3.5.2
2020-08-19 12:47:56 -04:00
dependabot-preview[bot]
a9e7f88887
Bump mockito-core from 3.5.0 to 3.5.2
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.5.0...v3.5.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 08:10:18 +00:00
dependabot-preview[bot]
aa07f4b21a
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200707-1.30.10 to v4-rev20200813-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 08:09:11 +00:00
Tom Morris
f74e3e048e
Fix i18n key typo - fixes #3093 (#3094) 2020-08-17 22:45:50 +02:00
Tom Morris
5eff85b76c
Update jQuery UI to 1.12 - fixes #3001 (#3010)
* Update jQuery UI from 1.10.3 to 1.12.1 and associated theme CSS

* Fix sidebar tab layout issue with new jQuery UI

* Update initialization jQuery UI Tabs widgets

selected is now active, but the first tab is selected by default
so we don't need to do it manually.

* Patch GData initialization error

Don't attempt to initialize if we get no docs back (ie unauthorized)
2020-08-17 14:49:56 +02:00
dependabot-preview[bot]
2045061ace
Bump mockito-core from 3.4.6 to 3.5.0 (#3092)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.6 to 3.5.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.6...v3.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:21:53 +02:00
dependabot-preview[bot]
0951af8646
Bump postgresql from 42.2.14 to 42.2.15 (#3091)
Bumps [postgresql](https://github.com/pgjdbc/pgjdbc) from 42.2.14 to 42.2.15.
- [Release notes](https://github.com/pgjdbc/pgjdbc/releases)
- [Changelog](https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pgjdbc/pgjdbc/compare/REL42.2.14...REL42.2.15)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:20:53 +02:00
Tom Morris
591d47abe3
Fix About page script references - fixes #3088 (#3090)
* Fix About page - fixes #3088

Update jQuery version
Include correct Javascript file to get version information
Fix version display

* Remove obsolete Freebase logo license reference
2020-08-17 13:16:11 +02:00
Tom Morris
fc21d58ed1
Don't count TABs as control characters - fixes #3061 (#3068)
* Don't count TABs as control characters - fixes #3061

* Add TSV test. Replace info logging w/assert message
2020-08-16 10:35:25 +02:00
Antonin Delpeuch
62ae8ae946
Merge pull request #2970 from OpenRefine/coc-update
Update code of conduct committee members
2020-08-15 10:41:19 +02:00
Tom Morris
7720cf2e1b
Merge pull request #3079 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-14 22:56:55 -04:00
Allan Nordhøy
e6197b8224
Translated using Weblate (Norwegian Bokmål)
Currently translated at 82.4% (169 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/nb_NO/
2020-08-15 00:32:54 +02:00
vuod
907b0d5169
Translated using Weblate (French)
Currently translated at 98.2% (738 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-08-15 00:32:54 +02:00
Antonin Delpeuch
18a34c51aa
Merge pull request #3076 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-14 10:57:40 +02:00
Hosted Weblate
43c1a28b4c
Merge branch 'origin/master' into Weblate. 2020-08-14 10:36:51 +02:00
dependabot-preview[bot]
0be85313ac
Bump jmh.version from 1.24 to 1.25 (#3077)
Bumps `jmh.version` from 1.24 to 1.25.

Updates `jmh-core` from 1.24 to 1.25

Updates `jmh-generator-annprocess` from 1.24 to 1.25

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-14 10:36:45 +02:00
Hosted Weblate
ef888ccdb2
Merge branch 'origin/master' into Weblate. 2020-08-13 18:16:12 +02:00
Shih-Chan Fan
97fe642ceb Translated using Weblate (Chinese (Traditional))
Currently translated at 6.3% (48 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/zh_Hant/
2020-08-13 12:16:06 -04:00
Shih-Chan Fan
6ab25d8fb6 Added translation using Weblate (Chinese (Traditional)) 2020-08-13 12:16:06 -04:00
Hosted Weblate
94e52d71eb
Merge branch 'origin/master' into Weblate. 2020-08-13 10:00:00 +02:00
Ekta Mishra
f50efb3699
Changed List return type to ImmutableList in Wikidata schema package classes (#3070)
* replaced ImmutableList to unmodifiableList

* added test cases to check unmodifiable Lists

* improved test cases
2020-08-13 09:59:54 +02:00
Hosted Weblate
4873c84293
Merge branch 'origin/master' into Weblate. 2020-08-12 18:16:45 +02:00
Shih-Chan Fan
cf0590b81f
Translated using Weblate (Chinese (Traditional))
Currently translated at 6.3% (48 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/zh_Hant/
2020-08-12 18:16:44 +02:00
Tom Morris
0ce5c5de13
Merge pull request #3065 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-resources-plugin-3.2.0
Bump maven-resources-plugin from 3.1.0 to 3.2.0
2020-08-12 12:16:38 -04:00
Shih-Chan Fan
c5d41266a5
Added translation using Weblate (Chinese (Traditional)) 2020-08-12 11:28:19 +02:00
dependabot-preview[bot]
937bb94a0e
Bump maven-resources-plugin from 3.1.0 to 3.2.0
Bumps [maven-resources-plugin](https://github.com/apache/maven-resources-plugin) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/apache/maven-resources-plugin/releases)
- [Commits](https://github.com/apache/maven-resources-plugin/compare/maven-resources-plugin-3.1.0...maven-resources-plugin-3.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-12 08:15:08 +00:00
Tom Morris
109aa78cf9
Merge pull request #3051 from OpenRefine/dependabot/maven/commons-validator-commons-validator-1.7
Bump commons-validator from 1.6 to 1.7
2020-08-10 13:04:12 -04:00
Tom Morris
b73b480d7d
Remove tests of third party code (#3050)
Neither of these tests are testing OpenRefine code
(and a negative NotEquals test is useless anyway)
2020-08-10 12:39:30 +02:00
dependabot-preview[bot]
1060a117ca
Bump commons-validator from 1.6 to 1.7
Bumps commons-validator from 1.6 to 1.7.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-10 08:19:45 +00:00
Antonin Delpeuch
e07baddce4
Merge pull request #3049 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-09 19:07:37 +02:00
Rafael Fontenelle
9662c0ab1a
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/pt_BR/
2020-08-09 18:32:49 +02:00
Allan Nordhøy
94da80ba3c
Translated using Weblate (Norwegian Bokmål)
Currently translated at 80.3% (49 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/nb_NO/
2020-08-09 18:32:49 +02:00
Rafael Fontenelle
680cfa996d
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/pt_BR/
2020-08-09 18:32:49 +02:00
Tom Morris
9c403d59d2
Add separator to zip slip check - fixes #3043 (#3048) 2020-08-09 14:48:55 +02:00
Tom Morris
55edae2b7b
Fix ToDate test failure & inefficiency - fixes #3026 (#3027)
* Fix ToDate test failure - fixes #3026

Instead of computing offset from UTC at current
point in time, use the offset from the parsed
date so that we're not affected by crossing
a daylight savings time boundary.

* Fix date parsing with locale as first format string

Also refactors for simpicity, restore some dropped tests,
and restores previous behavior of considering a bad
format string an error instead of silently ignoring it.

It does NOT address another issue which was introduced
in May 2018 of treating date/times without timzone
information as UTC instead of local.

* Restore error checking and messages

* Save & restore default timezone for tests

Also add some ToDos for places where LocalDate is being misused.
2020-08-09 13:53:43 +02:00
Antonin Delpeuch
6bcc2bfbe9
Merge pull request #3046 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-08 17:00:02 +02:00
Isao Matsunami
f8f25f39cd
Translated using Weblate (Japanese)
Currently translated at 100.0% (205 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-08-08 12:47:42 +02:00
Isao Matsunami
1b5a7d0e81
Translated using Weblate (Japanese)
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/ja/
2020-08-08 12:47:42 +02:00
Isao Matsunami
2f604e551a
Translated using Weblate (Japanese)
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-08-08 12:47:41 +02:00
Tom Morris
b09cde7002
Merge pull request #3045 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-07 20:45:33 -04:00
Hosted Weblate
478aeff793
Merge branch 'origin/master' into Weblate. 2020-08-08 02:20:33 +02:00
Tom Morris
c90f0f4cff
Translated using Weblate (French)
Currently translated at 99.0% (746 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-08-08 02:20:32 +02:00
Tom Morris
79399691a4
Translated using Weblate (Spanish)
Currently translated at 99.4% (749 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/es/
2020-08-08 02:20:32 +02:00