Commit Graph

5773 Commits

Author SHA1 Message Date
Antonin Delpeuch
9ac54edbba
Migrate reconciliation calls to Apache HTTP client (#2906)
* Migrate reconciliation calls to OkHTTP, for #2903

* Migrate to Apache HTTP Commons

* Migrate data extension to Apache HTTP client

* Deprecate HttpURLConnection in RefineServlet

* Use LaxRedirectStrategy, clean up imports

* Remove read and pool timeouts, only keep the connection timeout

* Adapt mocking of HTTP calls after migration
2020-08-23 14:04:59 +02:00
Tom Morris
259705ad5f
assertEqualAsJson test helper refactor (#3113)
* Refactor test helper

Create a version of the assert that uses the standard parameter
order and deprecate the version that uses inverted order.

* Use consistent Assert class and parameter ordering
2020-08-23 11:04:44 +02:00
Tom Morris
42e840dec6
Merge pull request #3110 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-22 12:17:37 -04:00
Isao Matsunami
9470c47031 Translated using Weblate (Japanese)
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-08-22 18:14:45 +02:00
Lu Liu
7d4e182c75
Extend Wikidata extension to support arbitrary Wikibase instances (#2810)
Closes #1640. All Wikibase-dependent parameters, which were previously hard-coded for Wikidata, are now described in a JSON manifest. The manifest is currently constructed by hand, but, in the future, will hopefully be published by each Wikibase instance at a standard location.

* setup the manifest framework

* add dependency mechanism to scrutinizers & update tests

* add json creators to constraint entities

* adapt the backend (units tests are to be updated)

* remove the call to prepareDependencies() in the constructor

* update code according to review feedback

* update scrutinizers tests

* fix typo & update ConstraintsV1

* log if a scrutinizer is skipped

* update versioning handling in the backend

* correct the order of "actual" and "expected" for assertEquals method

* use regex to check manifest versions

* 1. add wikibase-manager.js, wikibase-dialog.js, etc.
2. move dialog/schema-alignment-dialog.js -> schema-alignment.js
3. remove unused schema-alignment-dialog.html
4. change most mentions of "Wikidata" to "Wikibase"

* support saving cookies for different Wikibases & fix LoginCommandTest

* fix schema related tests

* removed unused WikibaseCredentials

* include MediaWiki API endpoint in the schema

* fetch language codes for different Wikibases

* fix lgtm-bot alerts

* keep a connection map (MediaWiki API endpoint => Connection) in ConnectionManager

* simplify the constraint configurations of the manifest and remove lots of unnecessary code.

* add slash to the end of mediawiki.root

* add manifest schema and use ajv to validate the manifest

* remove JSONP support (Wikibase manifest host should support CORS)

* save manifests on manifest update

* add unit tests for Manifest

* include the exception in logger.error() method to make it easier to debug

* include the message of ManifestException when calling respondError

* test multiple connections

* test no manifest & test invalid manifest

* adapt manage-account-dialog.js to support multiple Wikibase connections

* update instance/subclass of related translations

* beautify import-schema-dialog.html

* use "${lang}" variable in the reconciliation service endpoint of the manifest

* adapt schema-alignment.js after introducing "${lang}" variable in the reconciliation service endpoint

* use WikibaseManager.getSelectedWikibaseApi() in SchemaAlignment._getPropertyType

* replace more mentions of "Wikidata" to "Wikibase"

* use WikibaseManager.getSelectedWikibaseApi() in previewrenderer.js

* support fetching language codes of different Wikibases in the frontend

* skip EditInspector if missing 'property_constraint_pid' in the manifest

* improve unit tests for fetching lang codes

* skip scrutinizers depending on fetcher if 'property_constraint_pid' is missing in the manifest

* make sure the schema alignment panel is set up before rendering

* fix preview bug

* add getters of "instance of" and "subclass of" to the Manifest interface and use them in NewItemScrutinizer

* fix hardcode for Wikidata in WbItemVariable

* rename 'entity_prefix' to 'site_iri' and move it from 'manifest.wikibase.properties' to 'manifest.wikibase'

* include oauth configurations in the manifest & support logging in with owner-only consumer for Wikibases with the OAuth extension

* correct schema fallback logic

* select default wikibase according to the saved schema

* include maxlag in the manifest

* [backend] move maxlag setting from preferences to request parameter

* support setting maxlag when uploading edits

* rename "Manage Wikibase" to "Select Wikibase instance" and localize it

* fix manifest updating bug

* include EditGroups in the manifest

* add the reconciliation service from the manifest to standard services if it's not present yet when adding a new manifest

* update according to review feedback

1. use inherited color variable
2. rename 'gridwroks' to 'openrefine'
3. remove unnecessary 'async: true'
4. add 'format: url' validation to urls to the schema

* rename 'wikibasePrefix' to 'siteIri'
2020-08-22 11:58:56 -04:00
Antonin Delpeuch
68404abe2a
Fix race condition in service manifest fetching (#2872)
* Fix race condition in service manifest fetching. Fixes #2803

* Set own recon endpoint for Wikidata extension
2020-08-21 07:15:02 +02:00
Tom Morris
92ad819e34
Merge pull request #3100 from OpenRefine/dependabot/maven/org.mockito-mockito-core-3.5.2
Bump mockito-core from 3.5.0 to 3.5.2
2020-08-19 12:47:56 -04:00
dependabot-preview[bot]
a9e7f88887
Bump mockito-core from 3.5.0 to 3.5.2
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.5.0...v3.5.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 08:10:18 +00:00
dependabot-preview[bot]
aa07f4b21a
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200707-1.30.10 to v4-rev20200813-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 08:09:11 +00:00
Tom Morris
f74e3e048e
Fix i18n key typo - fixes #3093 (#3094) 2020-08-17 22:45:50 +02:00
Tom Morris
5eff85b76c
Update jQuery UI to 1.12 - fixes #3001 (#3010)
* Update jQuery UI from 1.10.3 to 1.12.1 and associated theme CSS

* Fix sidebar tab layout issue with new jQuery UI

* Update initialization jQuery UI Tabs widgets

selected is now active, but the first tab is selected by default
so we don't need to do it manually.

* Patch GData initialization error

Don't attempt to initialize if we get no docs back (ie unauthorized)
2020-08-17 14:49:56 +02:00
dependabot-preview[bot]
2045061ace
Bump mockito-core from 3.4.6 to 3.5.0 (#3092)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.6 to 3.5.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.6...v3.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:21:53 +02:00
dependabot-preview[bot]
0951af8646
Bump postgresql from 42.2.14 to 42.2.15 (#3091)
Bumps [postgresql](https://github.com/pgjdbc/pgjdbc) from 42.2.14 to 42.2.15.
- [Release notes](https://github.com/pgjdbc/pgjdbc/releases)
- [Changelog](https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pgjdbc/pgjdbc/compare/REL42.2.14...REL42.2.15)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:20:53 +02:00
Tom Morris
591d47abe3
Fix About page script references - fixes #3088 (#3090)
* Fix About page - fixes #3088

Update jQuery version
Include correct Javascript file to get version information
Fix version display

* Remove obsolete Freebase logo license reference
2020-08-17 13:16:11 +02:00
Tom Morris
fc21d58ed1
Don't count TABs as control characters - fixes #3061 (#3068)
* Don't count TABs as control characters - fixes #3061

* Add TSV test. Replace info logging w/assert message
2020-08-16 10:35:25 +02:00
Antonin Delpeuch
62ae8ae946
Merge pull request #2970 from OpenRefine/coc-update
Update code of conduct committee members
2020-08-15 10:41:19 +02:00
Tom Morris
7720cf2e1b
Merge pull request #3079 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-14 22:56:55 -04:00
Allan Nordhøy
e6197b8224
Translated using Weblate (Norwegian Bokmål)
Currently translated at 82.4% (169 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/nb_NO/
2020-08-15 00:32:54 +02:00
vuod
907b0d5169
Translated using Weblate (French)
Currently translated at 98.2% (738 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-08-15 00:32:54 +02:00
Antonin Delpeuch
18a34c51aa
Merge pull request #3076 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-14 10:57:40 +02:00
Hosted Weblate
43c1a28b4c
Merge branch 'origin/master' into Weblate. 2020-08-14 10:36:51 +02:00
dependabot-preview[bot]
0be85313ac
Bump jmh.version from 1.24 to 1.25 (#3077)
Bumps `jmh.version` from 1.24 to 1.25.

Updates `jmh-core` from 1.24 to 1.25

Updates `jmh-generator-annprocess` from 1.24 to 1.25

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-14 10:36:45 +02:00
Hosted Weblate
ef888ccdb2
Merge branch 'origin/master' into Weblate. 2020-08-13 18:16:12 +02:00
Shih-Chan Fan
97fe642ceb Translated using Weblate (Chinese (Traditional))
Currently translated at 6.3% (48 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/zh_Hant/
2020-08-13 12:16:06 -04:00
Shih-Chan Fan
6ab25d8fb6 Added translation using Weblate (Chinese (Traditional)) 2020-08-13 12:16:06 -04:00
Hosted Weblate
94e52d71eb
Merge branch 'origin/master' into Weblate. 2020-08-13 10:00:00 +02:00
Ekta Mishra
f50efb3699
Changed List return type to ImmutableList in Wikidata schema package classes (#3070)
* replaced ImmutableList to unmodifiableList

* added test cases to check unmodifiable Lists

* improved test cases
2020-08-13 09:59:54 +02:00
Hosted Weblate
4873c84293
Merge branch 'origin/master' into Weblate. 2020-08-12 18:16:45 +02:00
Shih-Chan Fan
cf0590b81f
Translated using Weblate (Chinese (Traditional))
Currently translated at 6.3% (48 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/zh_Hant/
2020-08-12 18:16:44 +02:00
Tom Morris
0ce5c5de13
Merge pull request #3065 from OpenRefine/dependabot/maven/org.apache.maven.plugins-maven-resources-plugin-3.2.0
Bump maven-resources-plugin from 3.1.0 to 3.2.0
2020-08-12 12:16:38 -04:00
Shih-Chan Fan
c5d41266a5
Added translation using Weblate (Chinese (Traditional)) 2020-08-12 11:28:19 +02:00
dependabot-preview[bot]
937bb94a0e
Bump maven-resources-plugin from 3.1.0 to 3.2.0
Bumps [maven-resources-plugin](https://github.com/apache/maven-resources-plugin) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/apache/maven-resources-plugin/releases)
- [Commits](https://github.com/apache/maven-resources-plugin/compare/maven-resources-plugin-3.1.0...maven-resources-plugin-3.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-12 08:15:08 +00:00
Tom Morris
109aa78cf9
Merge pull request #3051 from OpenRefine/dependabot/maven/commons-validator-commons-validator-1.7
Bump commons-validator from 1.6 to 1.7
2020-08-10 13:04:12 -04:00
Tom Morris
b73b480d7d
Remove tests of third party code (#3050)
Neither of these tests are testing OpenRefine code
(and a negative NotEquals test is useless anyway)
2020-08-10 12:39:30 +02:00
dependabot-preview[bot]
1060a117ca
Bump commons-validator from 1.6 to 1.7
Bumps commons-validator from 1.6 to 1.7.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-10 08:19:45 +00:00
Antonin Delpeuch
e07baddce4
Merge pull request #3049 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-09 19:07:37 +02:00
Rafael Fontenelle
9662c0ab1a
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/pt_BR/
2020-08-09 18:32:49 +02:00
Allan Nordhøy
94da80ba3c
Translated using Weblate (Norwegian Bokmål)
Currently translated at 80.3% (49 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/nb_NO/
2020-08-09 18:32:49 +02:00
Rafael Fontenelle
680cfa996d
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/pt_BR/
2020-08-09 18:32:49 +02:00
Tom Morris
9c403d59d2
Add separator to zip slip check - fixes #3043 (#3048) 2020-08-09 14:48:55 +02:00
Tom Morris
55edae2b7b
Fix ToDate test failure & inefficiency - fixes #3026 (#3027)
* Fix ToDate test failure - fixes #3026

Instead of computing offset from UTC at current
point in time, use the offset from the parsed
date so that we're not affected by crossing
a daylight savings time boundary.

* Fix date parsing with locale as first format string

Also refactors for simpicity, restore some dropped tests,
and restores previous behavior of considering a bad
format string an error instead of silently ignoring it.

It does NOT address another issue which was introduced
in May 2018 of treating date/times without timzone
information as UTC instead of local.

* Restore error checking and messages

* Save & restore default timezone for tests

Also add some ToDos for places where LocalDate is being misused.
2020-08-09 13:53:43 +02:00
Antonin Delpeuch
6bcc2bfbe9
Merge pull request #3046 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-08 17:00:02 +02:00
Isao Matsunami
f8f25f39cd
Translated using Weblate (Japanese)
Currently translated at 100.0% (205 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-08-08 12:47:42 +02:00
Isao Matsunami
1b5a7d0e81
Translated using Weblate (Japanese)
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/ja/
2020-08-08 12:47:42 +02:00
Isao Matsunami
2f604e551a
Translated using Weblate (Japanese)
Currently translated at 100.0% (751 of 751 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-08-08 12:47:41 +02:00
Tom Morris
b09cde7002
Merge pull request #3045 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-08-07 20:45:33 -04:00
Hosted Weblate
478aeff793
Merge branch 'origin/master' into Weblate. 2020-08-08 02:20:33 +02:00
Tom Morris
c90f0f4cff
Translated using Weblate (French)
Currently translated at 99.0% (746 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/fr/
2020-08-08 02:20:32 +02:00
Tom Morris
79399691a4
Translated using Weblate (Spanish)
Currently translated at 99.4% (749 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/es/
2020-08-08 02:20:32 +02:00
Tom Morris
1f5f0df624
Merge pull request #3044 from OpenRefine/dependabot/npm_and_yarn/docs/prismjs-1.21.0
Bump prismjs from 1.20.0 to 1.21.0 in /docs
2020-08-07 20:20:25 -04:00