Commit Graph

1003 Commits

Author SHA1 Message Date
Lu Liu
7d4e182c75
Extend Wikidata extension to support arbitrary Wikibase instances (#2810)
Closes #1640. All Wikibase-dependent parameters, which were previously hard-coded for Wikidata, are now described in a JSON manifest. The manifest is currently constructed by hand, but, in the future, will hopefully be published by each Wikibase instance at a standard location.

* setup the manifest framework

* add dependency mechanism to scrutinizers & update tests

* add json creators to constraint entities

* adapt the backend (units tests are to be updated)

* remove the call to prepareDependencies() in the constructor

* update code according to review feedback

* update scrutinizers tests

* fix typo & update ConstraintsV1

* log if a scrutinizer is skipped

* update versioning handling in the backend

* correct the order of "actual" and "expected" for assertEquals method

* use regex to check manifest versions

* 1. add wikibase-manager.js, wikibase-dialog.js, etc.
2. move dialog/schema-alignment-dialog.js -> schema-alignment.js
3. remove unused schema-alignment-dialog.html
4. change most mentions of "Wikidata" to "Wikibase"

* support saving cookies for different Wikibases & fix LoginCommandTest

* fix schema related tests

* removed unused WikibaseCredentials

* include MediaWiki API endpoint in the schema

* fetch language codes for different Wikibases

* fix lgtm-bot alerts

* keep a connection map (MediaWiki API endpoint => Connection) in ConnectionManager

* simplify the constraint configurations of the manifest and remove lots of unnecessary code.

* add slash to the end of mediawiki.root

* add manifest schema and use ajv to validate the manifest

* remove JSONP support (Wikibase manifest host should support CORS)

* save manifests on manifest update

* add unit tests for Manifest

* include the exception in logger.error() method to make it easier to debug

* include the message of ManifestException when calling respondError

* test multiple connections

* test no manifest & test invalid manifest

* adapt manage-account-dialog.js to support multiple Wikibase connections

* update instance/subclass of related translations

* beautify import-schema-dialog.html

* use "${lang}" variable in the reconciliation service endpoint of the manifest

* adapt schema-alignment.js after introducing "${lang}" variable in the reconciliation service endpoint

* use WikibaseManager.getSelectedWikibaseApi() in SchemaAlignment._getPropertyType

* replace more mentions of "Wikidata" to "Wikibase"

* use WikibaseManager.getSelectedWikibaseApi() in previewrenderer.js

* support fetching language codes of different Wikibases in the frontend

* skip EditInspector if missing 'property_constraint_pid' in the manifest

* improve unit tests for fetching lang codes

* skip scrutinizers depending on fetcher if 'property_constraint_pid' is missing in the manifest

* make sure the schema alignment panel is set up before rendering

* fix preview bug

* add getters of "instance of" and "subclass of" to the Manifest interface and use them in NewItemScrutinizer

* fix hardcode for Wikidata in WbItemVariable

* rename 'entity_prefix' to 'site_iri' and move it from 'manifest.wikibase.properties' to 'manifest.wikibase'

* include oauth configurations in the manifest & support logging in with owner-only consumer for Wikibases with the OAuth extension

* correct schema fallback logic

* select default wikibase according to the saved schema

* include maxlag in the manifest

* [backend] move maxlag setting from preferences to request parameter

* support setting maxlag when uploading edits

* rename "Manage Wikibase" to "Select Wikibase instance" and localize it

* fix manifest updating bug

* include EditGroups in the manifest

* add the reconciliation service from the manifest to standard services if it's not present yet when adding a new manifest

* update according to review feedback

1. use inherited color variable
2. rename 'gridwroks' to 'openrefine'
3. remove unnecessary 'async: true'
4. add 'format: url' validation to urls to the schema

* rename 'wikibasePrefix' to 'siteIri'
2020-08-22 11:58:56 -04:00
Antonin Delpeuch
68404abe2a
Fix race condition in service manifest fetching (#2872)
* Fix race condition in service manifest fetching. Fixes #2803

* Set own recon endpoint for Wikidata extension
2020-08-21 07:15:02 +02:00
dependabot-preview[bot]
a9e7f88887
Bump mockito-core from 3.5.0 to 3.5.2
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.5.0...v3.5.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 08:10:18 +00:00
Tom Morris
5eff85b76c
Update jQuery UI to 1.12 - fixes #3001 (#3010)
* Update jQuery UI from 1.10.3 to 1.12.1 and associated theme CSS

* Fix sidebar tab layout issue with new jQuery UI

* Update initialization jQuery UI Tabs widgets

selected is now active, but the first tab is selected by default
so we don't need to do it manually.

* Patch GData initialization error

Don't attempt to initialize if we get no docs back (ie unauthorized)
2020-08-17 14:49:56 +02:00
dependabot-preview[bot]
2045061ace
Bump mockito-core from 3.4.6 to 3.5.0 (#3092)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.6 to 3.5.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.6...v3.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:21:53 +02:00
dependabot-preview[bot]
0951af8646
Bump postgresql from 42.2.14 to 42.2.15 (#3091)
Bumps [postgresql](https://github.com/pgjdbc/pgjdbc) from 42.2.14 to 42.2.15.
- [Release notes](https://github.com/pgjdbc/pgjdbc/releases)
- [Changelog](https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pgjdbc/pgjdbc/compare/REL42.2.14...REL42.2.15)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-08-17 14:20:53 +02:00
Allan Nordhøy
e6197b8224
Translated using Weblate (Norwegian Bokmål)
Currently translated at 82.4% (169 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/nb_NO/
2020-08-15 00:32:54 +02:00
Ekta Mishra
f50efb3699
Changed List return type to ImmutableList in Wikidata schema package classes (#3070)
* replaced ImmutableList to unmodifiableList

* added test cases to check unmodifiable Lists

* improved test cases
2020-08-13 09:59:54 +02:00
dependabot-preview[bot]
937bb94a0e
Bump maven-resources-plugin from 3.1.0 to 3.2.0
Bumps [maven-resources-plugin](https://github.com/apache/maven-resources-plugin) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/apache/maven-resources-plugin/releases)
- [Commits](https://github.com/apache/maven-resources-plugin/compare/maven-resources-plugin-3.1.0...maven-resources-plugin-3.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-12 08:15:08 +00:00
Rafael Fontenelle
9662c0ab1a
Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/pt_BR/
2020-08-09 18:32:49 +02:00
Allan Nordhøy
94da80ba3c
Translated using Weblate (Norwegian Bokmål)
Currently translated at 80.3% (49 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/nb_NO/
2020-08-09 18:32:49 +02:00
Isao Matsunami
f8f25f39cd
Translated using Weblate (Japanese)
Currently translated at 100.0% (205 of 205 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-08-08 12:47:42 +02:00
Isao Matsunami
1b5a7d0e81
Translated using Weblate (Japanese)
Currently translated at 100.0% (61 of 61 strings)

Translation: OpenRefine/database
Translate-URL: https://hosted.weblate.org/projects/openrefine/database/ja/
2020-08-08 12:47:42 +02:00
dependabot-preview[bot]
267d1038a4
Bump google-api-services-drive
Bumps google-api-services-drive from v3-rev20200719-1.30.10 to v3-rev20200723-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-07 08:04:12 +00:00
Tom Morris
6095c44cb7
Update to jQuery 1.12.4 and jQuery Migrate 1.4.1 - fixes 2932 (#2933)
* Refactor module wiring to reduce redundancy

* Update to jQuery 1.12.4 & jQuery Migrate 1.4.1 - fixes #2932

This updates to the latest jQuery 1.x and jQuery Migrate 1.x,
the first step in upgrading to a modern jQuery.

* Add a couple of bug fixes from Google Code SVN

This is an unrelease version from the Google Code freebase-site
repo which only has a few changes from the v4.3 release, but
one of them is removing the `browser.msie` reference that
jQuery Migrate is complaining about.

* Use prop() for 'checked' and 'disabled'

* Update jQuery 'value' property setting code to use val()

* Use prop() instead of attr() to set 'selected'

* Patch for jQuery >1.9
2020-08-06 13:47:31 +02:00
Tom Morris
05fb4e355f
Replace Javascript string concatenation with i18n parameters - addresses #1858 (#3019)
* Replace js string concatenation with i18n parameters

refs #1858
Remove Javascript string concatentation and use jquery i18n()
instead so that translators have the needed context and
flexibility to be able to do a good job. Also remove code-based
plurals conditionalization and replace with i18n.

* Update French translation so I can test non-English support

* Add missing localization

* Clean up formatting of service API link
2020-08-04 10:10:38 +02:00
Tom Morris
2f7a357023
Merge pull request #3025 from OpenRefine/dependabot/maven/org.testng-testng-7.3.0
Bump testng from 7.1.0 to 7.3.0
2020-08-03 21:09:01 -04:00
Tom Morris
52194e1685 Add https for all TestNG DTDs 2020-08-03 12:27:58 -04:00
dependabot-preview[bot]
e081961c1f
Bump testng from 7.1.0 to 7.3.0
Bumps [testng](https://github.com/cbeust/testng) from 7.1.0 to 7.3.0.
- [Release notes](https://github.com/cbeust/testng/releases)
- [Changelog](https://github.com/cbeust/testng/blob/master/CHANGES.txt)
- [Commits](https://github.com/cbeust/testng/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-03 09:01:36 +00:00
dependabot-preview[bot]
1bebf73123
Bump google-api-services-drive
Bumps google-api-services-drive from v3-rev20200618-1.30.10 to v3-rev20200719-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-03 09:00:07 +00:00
Tom Morris
5985f95ce1
Explicitly set SameSite=Lax on cookie - fixes #3007 (#3014) 2020-08-01 10:58:16 +02:00
dependabot-preview[bot]
3210f14f33
Bump mockito-core from 3.4.4 to 3.4.6 (#3016)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.4 to 3.4.6.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.4...v3.4.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-30 14:50:32 +02:00
dependabot-preview[bot]
677ac408d6
Bump sqlite-jdbc from 3.32.3.1 to 3.32.3.2
Bumps [sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.32.3.1 to 3.32.3.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.32.3.1...3.32.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-29 08:17:07 +00:00
Ekta Mishra
fb6c309da8
Update EntityCache class (#2991)
* updated EntityCache class

* added overridden methods in CacheLoader and updated test file
2020-07-28 21:03:22 +02:00
dependabot-preview[bot]
f3ef39012e
Bump mariadb-java-client from 2.6.1 to 2.6.2
Bumps [mariadb-java-client](https://github.com/mariadb-corporation/mariadb-connector-j) from 2.6.1 to 2.6.2.
- [Release notes](https://github.com/mariadb-corporation/mariadb-connector-j/releases)
- [Changelog](https://github.com/mariadb-corporation/mariadb-connector-j/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mariadb-corporation/mariadb-connector-j/compare/2.6.1...2.6.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-24 08:06:28 +00:00
Tom Morris
83ed9ffdaf
Refactor importer APIs - Fixes #2963 (#2978)
* Make sure data directory is directory, not a file

* Add a test for zip archive import

Also tests the saving of the archive file name and source filename

* Add TODOs - no functional changes

* Cosmetic cleanups

* Revert importer API changes for archive file name parameter

Fixes #2963
- restore binary compatibility to the API
- hoist the handling of both fileSource and archiveFileName from
TabularImportingParserBase and TreeImportingParserBase to
ImportingParserBase so that there's only one copy. These 3 classes are
all part of the internal implementation, so there should be no
compatibility issue.

* Revert weird flow of control for import options metadata

This reverts the very convoluted control flow that was introduced
when adding the input options to the project metadata. Instead
the metadata is all handled in the importer framework rather than
having to change APIs are have individual importers worry about
it.

The feature never had test coverage, so that is still to be added.

* Add test for import options in project metadata & fix bug

Fixes bug where same options object was being reused and overwritten,
so all copies in the list ended up the same.
2020-07-23 18:36:14 +02:00
Tom Morris
c22078b2c9
Merge pull request #2953 from OpenRefine/dependabot/maven/com.google.apis-google-api-services-sheets-v4-rev20200707-1.30.10
Bump google-api-services-sheets from v4-rev20200616-1.30.10 to v4-rev20200707-1.30.10
2020-07-22 16:11:41 -04:00
Ekta Mishra
a4e5e49aaf
Add Citation needed scrutinizer (#2925)
Implemented Citation needed Cconstraint as part of #2354

Test class added with appropriate testc cases

Updated severity level to critical as well as the messages

merged unsourced and citation-needed scrutinizer

updated severity levels and warning messages
2020-07-21 08:09:37 +02:00
Ekta Mishra
6ab6bacd13
fixed some minor typos and refactoring in scrutinizer test files (#2958) 2020-07-20 20:58:15 +02:00
dependabot-preview[bot]
fa682d0e22
Bump mockito-core from 3.4.2 to 3.4.4 (#2965)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.2 to 3.4.4.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.2...v3.4.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-20 14:48:41 +02:00
Tom Morris
8cfdd5747a
Merge pull request #2952 from OpenRefine/dependabot/maven/org.mockito-mockito-core-3.4.2
Bump mockito-core from 3.3.3 to 3.4.2
2020-07-17 13:07:42 -04:00
Tom Morris
3e61565128
Merge pull request #2951 from OpenRefine/dependabot/maven/org.apache.commons-commons-lang3-3.11
Bump commons-lang3 from 3.10 to 3.11
2020-07-17 12:54:14 -04:00
Tom Morris
5336eca63e Remove redundant mockito dependencies 2020-07-17 12:48:06 -04:00
Hosted Weblate
032fd83048
Merge branch 'origin/master' into Weblate. 2020-07-17 12:41:56 +02:00
Allan Nordhøy
e0edbca1da
Translated using Weblate (Norwegian Bokmål)
Currently translated at 81.2% (165 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/nb_NO/
2020-07-17 12:41:52 +02:00
Isao Matsunami
fa8b9010e2
Translated using Weblate (Japanese)
Currently translated at 99.5% (202 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-17 12:41:51 +02:00
Allan Nordhøy
44f6e5ba64
Translated using Weblate (English)
Currently translated at 100.0% (203 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/en/
2020-07-17 12:41:51 +02:00
dependabot-preview[bot]
11600bf4cf
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200616-1.30.10 to v4-rev20200707-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:18:12 +00:00
dependabot-preview[bot]
edf8595977
Bump mockito-core from 3.3.3 to 3.4.2
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.3.3 to 3.4.2.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.3.3...v3.4.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:17:43 +00:00
dependabot-preview[bot]
0f391b5343
Bump commons-lang3 from 3.10 to 3.11
Bumps commons-lang3 from 3.10 to 3.11.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:17:12 +00:00
Ekta Mishra
dd17d7a2b6
Add test-cases for methods defined in WikidataConstraintFetcher (#2943)
* Add test-cases for methods defined in WikidataConstraintFetcher

tests for getConstraintsByTpe & getConstraintStatements methods.

* Tests updated
2020-07-16 18:34:26 +02:00
Hosted Weblate
0bfa8b7fd6
Merge branch 'origin/master' into Weblate. 2020-07-16 03:41:54 +02:00
Isao Matsunami
c78ff3be44
Translated using Weblate (Japanese)
Currently translated at 100.0% (201 of 201 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-16 03:41:50 +02:00
Tom Morris
2b5f3e30a6
Merge pull request #2942 from OpenRefine/dependabot/maven/org.xerial-sqlite-jdbc-3.32.3.1
Bump sqlite-jdbc from 3.32.3 to 3.32.3.1
2020-07-15 10:40:10 -04:00
Ekta Mishra
2bf493a498
Add One-of qualifier value property constraint (#2907)
* Add One-of qualifier value property constraint

Implemented one-of qualifier value property constraint as part of #2354

* Test class added

* Test cases updated and working fine

* resolved merge conflicts
2020-07-15 14:13:49 +02:00
Ekta Mishra
fe275ae634
Move findValues method to EditScrtuinzer (#2941)
* Move findValues method to EditScrtuinzer

As findValues method is not concerned with fetching, therefore moving it to EditScrtuinizer

* updated test classes accordingly

* updated test classes accordingly

* test files updated after shifting the findValues method
2020-07-15 11:46:06 +02:00
dependabot-preview[bot]
43785f81fd
Bump sqlite-jdbc from 3.32.3 to 3.32.3.1
Bumps [sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.32.3 to 3.32.3.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.32.3...3.32.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-15 08:18:33 +00:00
Hosted Weblate
f9d6c2b93b
Merge branch 'origin/master' into Weblate. 2020-07-14 22:41:55 +02:00
Susanna Ånäs
f01ca800bf
Translated using Weblate (Finnish)
Currently translated at 49.2% (95 of 193 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/fi/
2020-07-14 22:41:51 +02:00
Ekta Mishra
c7a8dfb329
Reuse Constraint IDs and Property IDs (#2937)
* Revert "Revert "Reused Constraint IDs and Parameter PIDs (#2893)""

This reverts commit 4c739be843.

* Reuse Constraint IDs and Parameter PIDs

After the cleanup of MockConstraintFetcher, using constraint IDs and property IDs defined in the corresponding scrtinizer for the test files.
2020-07-14 20:51:18 +02:00