Commit Graph

5211 Commits

Author SHA1 Message Date
Tom Morris
83ed9ffdaf
Refactor importer APIs - Fixes #2963 (#2978)
* Make sure data directory is directory, not a file

* Add a test for zip archive import

Also tests the saving of the archive file name and source filename

* Add TODOs - no functional changes

* Cosmetic cleanups

* Revert importer API changes for archive file name parameter

Fixes #2963
- restore binary compatibility to the API
- hoist the handling of both fileSource and archiveFileName from
TabularImportingParserBase and TreeImportingParserBase to
ImportingParserBase so that there's only one copy. These 3 classes are
all part of the internal implementation, so there should be no
compatibility issue.

* Revert weird flow of control for import options metadata

This reverts the very convoluted control flow that was introduced
when adding the input options to the project metadata. Instead
the metadata is all handled in the importer framework rather than
having to change APIs are have individual importers worry about
it.

The feature never had test coverage, so that is still to be added.

* Add test for import options in project metadata & fix bug

Fixes bug where same options object was being reused and overwritten,
so all copies in the list ended up the same.
2020-07-23 18:36:14 +02:00
Tom Morris
7512f3cb52 Update to Butterfly 1.0.4 - fixes #2917
Fixes #2917
Update to Butterfly 1.0.4 which catches NoClassDefFound errors
for Butterfly modules (ie OpenRefine extensions) which are missing
Java dependencies (e.g. those built against earlier versions
of OpenRefine)
2020-07-23 12:21:24 -04:00
Tom Morris
d5abaac6df
Update marc4j to 2.9.1 - Fixes #2962 (#2977)
* Add a MARC import test

* Make sure data directory is directory, not a file

* Update to marc4j 2.9.1 - fixes #2962
2020-07-22 22:12:30 +02:00
Tom Morris
c22078b2c9
Merge pull request #2953 from OpenRefine/dependabot/maven/com.google.apis-google-api-services-sheets-v4-rev20200707-1.30.10
Bump google-api-services-sheets from v4-rev20200616-1.30.10 to v4-rev20200707-1.30.10
2020-07-22 16:11:41 -04:00
allanaaa
e8ead8dfa7
Edits to Installing page (#2908)
* Edits to Installing page

Testing a wide variety of questionable formatting options.

* Testing docusaurus tip boxes

* Testing more

* Tip boxes are ... bright, aren't they

* Formatting fixes

* Update installing.md

* Update installing.md

* Adding memory allocation section and doing some more formatting experiments

* Update installing.md

* typo!

* Update installing.md

* Update installing.md

* Testing tabs

Also I was partway through writing Linux instructions.

* Testing tabs

First commit didn't go through?

* MD to MDX test

* Update installing.mdx

* Changing back!

* Remove extra line breaks

* Update installing.md

formatting and link updates and etc

* Update installing.md

* Adding horizontal lines

* More line-break issues

* Adding images

* Whoops, forgot how images work

Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
2020-07-21 17:52:00 +02:00
Owen Stephens
a312859fb1
Remove image file from incorrect location (#2972) 2020-07-21 14:39:23 +02:00
Ekta Mishra
a4e5e49aaf
Add Citation needed scrutinizer (#2925)
Implemented Citation needed Cconstraint as part of #2354

Test class added with appropriate testc cases

Updated severity level to critical as well as the messages

merged unsourced and citation-needed scrutinizer

updated severity levels and warning messages
2020-07-21 08:09:37 +02:00
Ekta Mishra
6ab6bacd13
fixed some minor typos and refactoring in scrutinizer test files (#2958) 2020-07-20 20:58:15 +02:00
dependabot-preview[bot]
fa682d0e22
Bump mockito-core from 3.4.2 to 3.4.4 (#2965)
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.4.2 to 3.4.4.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.4.2...v3.4.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-20 14:48:41 +02:00
Tom Morris
8cfdd5747a
Merge pull request #2952 from OpenRefine/dependabot/maven/org.mockito-mockito-core-3.4.2
Bump mockito-core from 3.3.3 to 3.4.2
2020-07-17 13:07:42 -04:00
Tom Morris
3e61565128
Merge pull request #2951 from OpenRefine/dependabot/maven/org.apache.commons-commons-lang3-3.11
Bump commons-lang3 from 3.10 to 3.11
2020-07-17 12:54:14 -04:00
Tom Morris
5336eca63e Remove redundant mockito dependencies 2020-07-17 12:48:06 -04:00
Antonin Delpeuch
1dcc4aadc4
Merge pull request #2955 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-17 12:52:25 +02:00
Hosted Weblate
032fd83048
Merge branch 'origin/master' into Weblate. 2020-07-17 12:41:56 +02:00
Allan Nordhøy
e0edbca1da
Translated using Weblate (Norwegian Bokmål)
Currently translated at 81.2% (165 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/nb_NO/
2020-07-17 12:41:52 +02:00
Isao Matsunami
fa8b9010e2
Translated using Weblate (Japanese)
Currently translated at 99.5% (202 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-17 12:41:51 +02:00
Allan Nordhøy
44f6e5ba64
Translated using Weblate (English)
Currently translated at 100.0% (203 of 203 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/en/
2020-07-17 12:41:51 +02:00
dependabot-preview[bot]
11600bf4cf
Bump google-api-services-sheets
Bumps google-api-services-sheets from v4-rev20200616-1.30.10 to v4-rev20200707-1.30.10.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:18:12 +00:00
dependabot-preview[bot]
edf8595977
Bump mockito-core from 3.3.3 to 3.4.2
Bumps [mockito-core](https://github.com/mockito/mockito) from 3.3.3 to 3.4.2.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](https://github.com/mockito/mockito/compare/v3.3.3...v3.4.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:17:43 +00:00
dependabot-preview[bot]
0f391b5343
Bump commons-lang3 from 3.10 to 3.11
Bumps commons-lang3 from 3.10 to 3.11.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-17 08:17:12 +00:00
Owen Stephens
a0175cd262
Merge pull request #2950 from OpenRefine/thadguidry-docs-tech
Minor corrections to tech docs to test editing
2020-07-16 20:28:31 +01:00
Thad Guidry
22d10f32d5
Minor corrections to tech docs to test editing 2020-07-16 13:33:08 -05:00
Owen Stephens
09add9fa31
Added technical reference to docusaurus based on existing wiki pages (#2940) 2020-07-16 18:39:03 +02:00
Ekta Mishra
dd17d7a2b6
Add test-cases for methods defined in WikidataConstraintFetcher (#2943)
* Add test-cases for methods defined in WikidataConstraintFetcher

tests for getConstraintsByTpe & getConstraintStatements methods.

* Tests updated
2020-07-16 18:34:26 +02:00
Tom Morris
e67c6c9580
Merge pull request #2949 from OpenRefine/dependabot/npm_and_yarn/docs/lodash-4.17.19
Bump lodash from 4.17.15 to 4.17.19 in /docs
2020-07-16 11:02:10 -04:00
Antonin Delpeuch
858bd463a4 Revert "Bump git-commit-id-plugin from 4.0.0 to 4.0.1 (#2948)"
The dependency update broke the snapshot release process on GitHub Actions:
https://github.com/OpenRefine/OpenRefine/runs/876878262

This reverts commit 52bb2c4d38.
2020-07-16 11:07:54 +02:00
dependabot[bot]
05c0f31ef0
Bump lodash from 4.17.15 to 4.17.19 in /docs
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-16 09:02:29 +00:00
dependabot-preview[bot]
52bb2c4d38
Bump git-commit-id-plugin from 4.0.0 to 4.0.1 (#2948)
Bumps git-commit-id-plugin from 4.0.0 to 4.0.1.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-16 11:01:43 +02:00
Tom Morris
f2e61b6628
Add tests for wide XLS/XLSX export (#2945)
Refs #2122. Also reenable a couple of disabled tests
2020-07-16 10:01:17 +02:00
Tom Morris
67801e68b4
Merge pull request #2947 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-15 22:38:36 -04:00
Hosted Weblate
0bfa8b7fd6
Merge branch 'origin/master' into Weblate. 2020-07-16 03:41:54 +02:00
Isao Matsunami
c78ff3be44
Translated using Weblate (Japanese)
Currently translated at 100.0% (201 of 201 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/ja/
2020-07-16 03:41:50 +02:00
Tom Morris
b2f76ca49d
More bug template tweaks (#2946)
- Motivating change is removal of boilerplate text before description so we can see more of the description in hover previews.
- Also changes section headings from bolded to headings 
- And combines OpenRefine version in same section as other versions
2020-07-15 21:48:08 +02:00
Tom Morris
2b5f3e30a6
Merge pull request #2942 from OpenRefine/dependabot/maven/org.xerial-sqlite-jdbc-3.32.3.1
Bump sqlite-jdbc from 3.32.3 to 3.32.3.1
2020-07-15 10:40:10 -04:00
Ekta Mishra
2bf493a498
Add One-of qualifier value property constraint (#2907)
* Add One-of qualifier value property constraint

Implemented one-of qualifier value property constraint as part of #2354

* Test class added

* Test cases updated and working fine

* resolved merge conflicts
2020-07-15 14:13:49 +02:00
Ekta Mishra
fe275ae634
Move findValues method to EditScrtuinzer (#2941)
* Move findValues method to EditScrtuinzer

As findValues method is not concerned with fetching, therefore moving it to EditScrtuinizer

* updated test classes accordingly

* updated test classes accordingly

* test files updated after shifting the findValues method
2020-07-15 11:46:06 +02:00
dependabot-preview[bot]
43785f81fd
Bump sqlite-jdbc from 3.32.3 to 3.32.3.1
Bumps [sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.32.3 to 3.32.3.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.32.3...3.32.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-15 08:18:33 +00:00
Tom Morris
a3fab26cca
Fix the text format guesser so it doesn't inappropriately guess WikiText (#2924)
* Fix text guesser so it doesn't guess wikitext

Fixes #2850
- Add simple magic detector for zip & gzip files to keep
  it from attempting to guess binary files
- Add a counter for C0 controls for the same reason
- Tighten wikitable counters to require marker at
  beginning of the line, per the specification
- Refactor to use Apache Commons instead of private
  counting methods
- Add tests for most TextGuesser formats

* Remove misplaced duplicate test data file

* Fix LGTM warning + minor cleanups

* Use BoundedInputStream to prevent runaway lines
2020-07-15 08:56:00 +02:00
Antonin Delpeuch
fb9c8e5fef
Merge pull request #2939 from weblate/weblate-openrefine-translations
Translations update from Weblate
2020-07-15 00:15:58 +02:00
Hosted Weblate
f9d6c2b93b
Merge branch 'origin/master' into Weblate. 2020-07-14 22:41:55 +02:00
Susanna Ånäs
f01ca800bf
Translated using Weblate (Finnish)
Currently translated at 49.2% (95 of 193 strings)

Translation: OpenRefine/wikidata
Translate-URL: https://hosted.weblate.org/projects/openrefine/wikidata/fi/
2020-07-14 22:41:51 +02:00
Isao Matsunami
a2100f64f7
Translated using Weblate (Japanese)
Currently translated at 100.0% (753 of 753 strings)

Translation: OpenRefine/Translations
Translate-URL: https://hosted.weblate.org/projects/openrefine/translations/ja/
2020-07-14 22:41:51 +02:00
Johan Richer
5ac4362c86
Fix FAQ URL (#2938) 2020-07-14 20:51:53 +02:00
Ekta Mishra
c7a8dfb329
Reuse Constraint IDs and Property IDs (#2937)
* Revert "Revert "Reused Constraint IDs and Parameter PIDs (#2893)""

This reverts commit 4c739be843.

* Reuse Constraint IDs and Parameter PIDs

After the cleanup of MockConstraintFetcher, using constraint IDs and property IDs defined in the corresponding scrtinizer for the test files.
2020-07-14 20:51:18 +02:00
Antonin Delpeuch
4c739be843 Revert "Reused Constraint IDs and Parameter PIDs (#2893)"
This introduces a build failure since other refactorings took place.

This reverts commit f41ce3a752.
2020-07-14 20:15:59 +02:00
Ekta Mishra
f41ce3a752
Reused Constraint IDs and Parameter PIDs (#2893)
Reused the hardcodedd constraint & paramter IDs from the scrutinizer classes into the test classes repectively.
2020-07-14 20:12:46 +02:00
Tom Morris
561619399c
Fix order dependent NPE in LoadLanguage test (#2922)
* Ensure ProjectManager is initialized before test - fixes #2895

* Fix indentation (detabify)
2020-07-14 18:06:04 +02:00
Ekta Mishra
c5e6ac9f90
Remove unused scrutinizer specific methods (#2935)
After adding the code for fetching parameters for different scrutinizers in their corresponding classes, we should remove the redundant code from the ConstraintFetcher
2020-07-14 17:51:06 +02:00
Tom Morris
ed68541988
Remove informational logging from tests that are passing (#2923)
* Change logging from info to debug

* Make tests less chatty when they're passing
2020-07-14 17:47:36 +02:00
Urvashi Gupta
f00129b852
fixes service panel toggling (#2915) 2020-07-14 16:46:10 +02:00