Fix 3.5 docs
This commit is contained in:
Antonin Delpeuch
parent
f66c7d2e5c
commit
d6b5e257dc
@ -6,7 +6,7 @@ sidebar_label: Introduction
|
||||
---
|
||||
|
||||
|
||||
This manual is designed to comprehensively walk through every aspect of setting up and using OpenRefine 3.4.1, including every interface function and feature.
|
||||
This manual is designed to comprehensively walk through every aspect of setting up and using OpenRefine 3.5.0, including every interface function and feature.
|
||||
|
||||
<!--
|
||||
This documentation platform provides a separate version of the user manual for each version of OpenRefine (from 3.4.1 onwards) - if you're looking for a later version than 3.4.1, please select the correct version from the dropdown menu in the top bar of this page.
|
||||
|
||||
@ -29,9 +29,9 @@ To export data from a project, click the <span class="menuItems">Export</span> d
|
||||
|
||||
You can also export reconciled data to Wikidata, or export your Wikidata schema for future use with other OpenRefine projects:
|
||||
|
||||
* [Upload edits to Wikidata](wikidata#upload-edits-to-wikidata)
|
||||
* [Export to QuickStatements](wikidata#quickstatements-export) (version 1)
|
||||
* [Export Wikidata schema](wikidata#import-and-export-schema)
|
||||
* [Upload edits to Wikidata](wikibase/uploading#uploading-with-openrefine)
|
||||
* [Export to QuickStatements](wikibase/uploading#uploading-with-quickstatements) (version 1)
|
||||
* [Export Wikidata schema](wikibase/overview#import-and-export-schema)
|
||||
|
||||
### Custom tabular exporter {#custom-tabular-exporter}
|
||||
|
||||
|
||||
@ -92,12 +92,12 @@ Do not use slashes to wrap regular expressions outside of a GREL expression.
|
||||
|
||||
On the [GREL functions](grelfunctions) page, functions that support regex will indicate that with a “p” for “pattern.” The GREL functions that support regex are:
|
||||
* [contains](grelfunctions#containss-sub-or-p)
|
||||
* [replace](grelfunctions#replaces-s-or-p-find-s-replace)
|
||||
* [find](grelfunctions#finds-sub-or-p)
|
||||
* [replace](grelfunctions#find-and-replace)
|
||||
* [find](grelfunctions#find-and-replace)
|
||||
* [match](grelfunctions#matchs-p)
|
||||
* [partition](grelfunctions#partitions-s-or-p-fragment-b-omitfragment-optional)
|
||||
* [rpartition](grelfunctions#rpartitions-s-or-p-fragment-b-omitfragment-optional)
|
||||
* [split](grelfunctions#splits-s-or-p-sep)
|
||||
* [split](grelfunctions#splits-s-or-p-sep-b-preservetokens-optional)
|
||||
* [smartSplit](grelfunctions#smartsplits-s-or-p-sep-optional)
|
||||
|
||||
### Jython-supported regex {#jython-supported-regex}
|
||||
|
||||
@ -100,6 +100,10 @@ You will be offered the option to include blank, non-numeric, and error values i
|
||||
You can create a text facet on numeric data, which will treat each entry as a string. This can be useful if you wish, for example, to manually include facets instead of selecting a range, or sort by count, or copy that count.
|
||||
:::
|
||||
|
||||
:::info Faceting customization
|
||||
As mentioned in the overview, facets can be modified or customized by GREL [expressions](expressions) in many ways. For example, to facet by clusters of [row](expressions#variables) numbers with `row.index/100` or better visualizing numbers greater than 1000 with `max(row.index, 1000)`.
|
||||
:::
|
||||
|
||||
## Timeline facet {#timeline-facet}
|
||||
|
||||
|
||||
|
||||
@ -52,7 +52,7 @@ You can use toString() to convert numbers to strings with rounding, using an [op
|
||||
|0.15|0|
|
||||
|100.0|100|
|
||||
|
||||
You can also convert dates to strings, using date parsing syntax built in to OpenRefine (see [the toDate() function for details](#todateo-boolean-month_first--format1-format2--)). For example, `value.toString("MMM-dd-yyyy")` would convert the date value [2024-10-15T00:00:00Z] to “Oct-15-2024”.
|
||||
You can also convert dates to strings, using date parsing syntax built into OpenRefine (see [the toDate() function for details](#todateo-b-monthfirst-s-format1-s-format2-)). For example, `value.toString("MMM-dd-yyyy")` would convert the date value [2024-10-15T00:00:00Z] to “Oct-15-2024”.
|
||||
|
||||
Note: In OpenRefine, using toString() on a null cell outputs the string “null”.
|
||||
|
||||
|
||||
@ -20,7 +20,7 @@ OpenRefine is designed to work with **Windows**, **Mac**, and **Linux** operatin
|
||||
|
||||
If you install and start OpenRefine on a Windows computer without Java, it will automatically open up a browser window to the [Java downloads page](https://java.com/en/download/), and you can simply follow the instructions there.
|
||||
|
||||
We recommend you [download](https://java.com/en/download/) and install Java before proceeding with the OpenRefine installation.
|
||||
We recommend you [download](https://java.com/en/download/) and install Java before proceeding with the OpenRefine installation. Please note that OpenRefine works with Java 8 to Java 15 but not Java 16 or later versions.
|
||||
|
||||
#### Compatible browser {#compatible-browser}
|
||||
|
||||
@ -49,7 +49,7 @@ For the absolute latest development updates, see the [snapshot releases](https:/
|
||||
|
||||
#### What’s changed {#whats-changed}
|
||||
|
||||
Our [latest version is OpenRefine 3.4.1](https://github.com/OpenRefine/OpenRefine/releases/tag/3.4.1), released September 24th 2020. The major changes in this version are listed on the [3.4.1 release page](https://github.com/OpenRefine/OpenRefine/releases/tag/3.4.1) with the downloadable packages.
|
||||
Our [latest version is OpenRefine 3.5.0](https://github.com/OpenRefine/OpenRefine/releases/tag/3.5.0), released November 7th 2021. The major changes in this version are listed on the [3.5.0 release page](https://github.com/OpenRefine/OpenRefine/releases/tag/3.5.0) with the downloadable packages.
|
||||
|
||||
You can find information about all OpenRefine versions on the [Releases page on Github](https://github.com/OpenRefine/OpenRefine/releases).
|
||||
|
||||
@ -187,7 +187,7 @@ If you had previously installed the `openrefine-dev` cask (containing a release
|
||||
Once you have downloaded the `.tar.gz` file, open a shell, navigate to the folder containing the download, and type:
|
||||
|
||||
```
|
||||
tar xzf openrefine-linux-3.4.tar.gz
|
||||
tar xzf openrefine-linux-3.5.0.tar.gz
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
@ -6,7 +6,7 @@ sidebar_label: Reconciling
|
||||
|
||||
## Overview {#overview}
|
||||
|
||||
Reconciliation is the process of matching your dataset with that of an external source. Datasets for comparison might be produced by libraries, archives, museums, academic organizations, scientific institutions, non-profits, or interest groups. You can also reconcile against user-edited data on [Wikidata](wikidata), or reconcile against [a local dataset that you yourself supply](https://github.com/OpenRefine/OpenRefine/wiki/Reconcilable-Data-Sources#local-services).
|
||||
Reconciliation is the process of matching your dataset with that of an external source. Datasets for comparison might be produced by libraries, archives, museums, academic organizations, scientific institutions, non-profits, or interest groups. You can also reconcile against user-edited data on [Wikidata or other Wikibase instances](wikibase/reconciling), or reconcile against [a local dataset that you yourself supply](https://github.com/OpenRefine/OpenRefine/wiki/Reconcilable-Data-Sources#local-services).
|
||||
|
||||
To reconcile your OpenRefine project against an external dataset, that dataset must offer a web service that conforms to the [Reconciliation Service API standards](https://reconciliation-api.github.io/specs/0.1/).
|
||||
|
||||
@ -31,7 +31,7 @@ OpenRefine maintains a [further list of sources on the wiki](https://github.com/
|
||||
|
||||
Other services may exist that are not yet listed in these two places: for example, the [310 datasets hosted by the Organized Crime and Corruption Reporting Project (OCCRP)](https://aleph.occrp.org/datasets/) each have their own reconciliation URL, or you can reconcile against their entire database with the URL [shared on the reconciliation API list](https://reconciliation-api.github.io/testbench/). For another example, you can reconcile against the entire Virtual International Authority File (VIAF) dataset, or [only the contributions from certain institutions](http://refine.codefork.com/). Search online to see if the authority you wish to reconcile against has an available service, or whether you can download a copy to reconcile against locally.
|
||||
|
||||
OpenRefine includes Wikidata reconciliation in the installation package - see the [Wikidata](wikidata) page for more information particular to that service. Extensions can add reconciliation services, and can also add enhanced reconciliation capacities. Check the list of extensions on the [Downloads page](https://openrefine.org/download.html) for more information.
|
||||
OpenRefine includes Wikidata reconciliation in the installation package - see the [Wikibase](wikibase/reconciling) page for more information particular to that service. Extensions can add reconciliation services, and can also add enhanced reconciliation capacities. Check the list of extensions on the [Downloads page](https://openrefine.org/download.html) for more information.
|
||||
|
||||
Each source will have its own documentation on how it provides reconciliation. The table on [the reconciliation API list](https://reconciliation-api.github.io/testbench/) indicates whether your chosen service supports the features described below. Refer to the service's documentation if you have questions about its behaviors and which OpenRefine features it supports.
|
||||
|
||||
@ -75,7 +75,7 @@ Hovering over the suggestion will also offer the two matching options as buttons
|
||||
|
||||
For matched values (those appearing as dark blue links), the underlying cell value has not been altered - the cell is storing both the original string and the matched entity link at the same time. If you were to copy your column to a new column at this point using `value`, for example, the reconcilation data would not transfer - only the original strings. You can learn more about how OpenRefine stores different pieces of information in each cell in [the Variables section specific to reconciliation data](expressions#reconciliation).
|
||||
|
||||
For each cell, you can manually “Create new item,” which will take the cell’s original value and apply it, as though it is a match. This will not become a dark blue link, because at this time there is nothing to link to: it is a draft entity stored only in your project. You can use this feature to prepare these entries for eventual upload to an editable service such as [Wikidata](wikidata), but most services do not yet support this feature.
|
||||
For each cell, you can manually “Create new item,” which will take the cell’s original value and apply it, as though it is a match. This will not become a dark blue link, because at this time there is nothing to link to: it is a draft entity stored only in your project. You can use this feature to prepare these entries for eventual upload to an editable service such as [Wikibase](wikibase/overview), but most services do not yet support this feature.
|
||||
|
||||
### Reconciliation facets {#reconciliation-facets}
|
||||
|
||||
@ -237,6 +237,6 @@ You can find out more in the [reconciliaton variables](expressions#reconciliaton
|
||||
|
||||
## Exporting reconciled data {#exporting-reconciled-data}
|
||||
|
||||
Once you have data that is reconciled to existing entities online, you may wish to export that data to a user-editable service such as Wikidata. See the section on [uploading your edits to Wikidata](wikidata#upload-edits-to-wikidata) for more information, or the section on [exporting](exporting) to see other formats OpenRefine can produce.
|
||||
Once you have data that is reconciled to existing entities online, you may wish to export that data to a user-editable service such as Wikidata. See the section on [uploading your edits to Wikidata or other Wikibase instances](wikibase/uploading) for more information, or the section on [exporting](exporting) to see other formats OpenRefine can produce.
|
||||
|
||||
You can share reconciled data in progress through a [project export or import](exporting#export-a-project), with some preparation. The importing user needs to have the appropriate reconciliation services installed on their OpenRefine instance (by going to <span class="menuItems">Start reconciling</span> and clicking on <span class="buttonLabels">Add Standard Service...</span>) in advance of opening the project, in order to use candidate and match links. Otherwise, the links will be broken and the user will need to add the reconciliation service and re-reconcile the columns in question. [Wikidata](wikidata) reconciliation data can be shared more easily as the service comes bundled with OpenRefine.
|
||||
You can share reconciled data in progress through a [project export or import](exporting#export-a-project), with some preparation. The importing user needs to have the appropriate reconciliation services installed on their OpenRefine instance (by going to <span class="menuItems">Start reconciling</span> and clicking on <span class="buttonLabels">Add Standard Service...</span>) in advance of opening the project, in order to use candidate and match links. Otherwise, the links will be broken and the user will need to add the reconciliation service and re-reconcile the columns in question. [Wikidata](wikibase/reconciling) reconciliation data can be shared more easily as the service comes bundled with OpenRefine.
|
||||
|
||||
@ -7,7 +7,7 @@ explains how to work around these cases.
|
||||
Sometimes your source provides data in a structured format, such as XML,
|
||||
JSON or RDF. OpenRefine can import these files and will convert them to
|
||||
tables. These tables will reflect some of the hierarchy in the file by
|
||||
means of null cells, using the [records mode](manual/exploring#rows-vs-records).
|
||||
means of null cells, using the [records mode](/manual/exploring#rows-vs-records).
|
||||
|
||||
The Wikibase extension always works in rows mode, so if we want to add
|
||||
statements which reference both the artist and the song, we need to fill
|
||||
|
||||
@ -36,7 +36,7 @@ Do not worry about the ordering of items in the schema or the order of
|
||||
your rows, as OpenRefine will rearrange your edits to optimize their
|
||||
upload. If your project makes edits on the same item across multiple
|
||||
rows, these edits will be merged together and performed in one edit. See
|
||||
[Uploading your changes](./upload) about that.
|
||||
[Uploading your changes](./uploading) about that.
|
||||
|
||||
## Terms {#terms}
|
||||
|
||||
|
||||
@ -15,6 +15,10 @@ You will need:
|
||||
|
||||
From the top level directory in the OpenRefine application you can build, test and run OpenRefine using the `./refine` shell script (if you are working in a \*nix shell), or using the `refine.bat` script from the Windows command line. Note that the `refine.bat` on Windows only supports a subset of the functionality, supported by the `refine` shell script. The example commands below are using the `./refine` shell script, and you will need to use `refine.bat` if you are working from the Windows command line.
|
||||
|
||||
### Get OpenRefine source code
|
||||
|
||||
With Git installed, use the `git clone` command to download the [project's repo](https://github.com/OpenRefine/OpenRefine) to a directory of your choice.
|
||||
|
||||
### Set up JDK {#set-up-jdk}
|
||||
|
||||
You must [install JDK](https://jdk.java.net/15/) and set the JAVA_HOME environment variable (please ensure it points to the JDK, and not the JRE).
|
||||
@ -111,6 +115,8 @@ export JAVA_HOME
|
||||
export PATH
|
||||
```
|
||||
|
||||
Note: OpenRefine on Linux currently supports jdk versions 8 to 15. Reference: [Issue 4106](https://github.com/OpenRefine/OpenRefine/issues/4106).
|
||||
|
||||
Save and close the file. When you are back in the terminal, type
|
||||
|
||||
```
|
||||
|
||||
@ -61,7 +61,7 @@ This describes the overall steps to your first code contribution in OpenRefine.
|
||||
|
||||
- Make changes to the code to fix the issue. If you are changing backend code, it would be great if you could also write a test in Java to demonstrate the fix. You can imitate existing tests for that. We currently do not have frontend tests.
|
||||
|
||||
- commit your changes, using a message that contains "closes #1234" or "fixes #1234", this will link the commit to the issue you are working on.
|
||||
- commit your changes, using a message that contains one of the special words "closes" and "fixes" which are detected by Github, followed by the issue number, e.g. "closes #1234" or "fixes #1234", this will link the commit to the issue you are working on.
|
||||
|
||||
- push your branch to your fork and create a pull request for it, explaining the approach you have used, any design decisions you have made.
|
||||
|
||||
|
||||
@ -179,6 +179,8 @@ Cypress execution can be configured with environment variables, they can be decl
|
||||
Available variables are
|
||||
|
||||
- OPENREFINE_URL, determine on which scheme://url:port to access OpenRefine, default to http://localhost:333
|
||||
- DISABLE_PROJECT_CLEANUP, If set to 1, projects will not be deleted after each run. Default to 0 to keep the OpenRefine instance clean
|
||||
|
||||
|
||||
Cypress contains [exaustive documentation](https://docs.cypress.io/guides/guides/environment-variables.html#Setting) about configuration, but here are two simple ways to configure the execution of the tests:
|
||||
|
||||
|
||||
@ -1,105 +0,0 @@
|
||||
---
|
||||
id: translating
|
||||
title: Translate the OpenRefine interface
|
||||
sidebar_label: Translate the OpenRefine interface
|
||||
---
|
||||
|
||||
Currently supported languages include English, Spanish, Chinese, French, Hebrew, Italian and Japanese.
|
||||
|
||||
|
||||
|
||||
You can help translate OpenRefine into your language by visiting [Weblate](https://hosted.weblate.org/engage/openrefine/?utm_source=widget) which provides a web based UI to edit and add translations and sends automatic pull requests back to our project.
|
||||
|
||||
Click to help translate --> [Weblate](https://hosted.weblate.org/engage/openrefine/?utm_source=widget)
|
||||
|
||||
## User entry of language data ## {#user-entry-of-language-data-}
|
||||
|
||||
Localized strings are entered in a .json file, one per language. They are located in the folder `main/webapp/modules/core/langs/` in a file named `translation-xx`.json, where xx is the language code (i.e. fr for French).
|
||||
|
||||
### Simple case of localized string ### {#simple-case-of-localized-string-}
|
||||
This is an example of a simple string, with the start of the JSON file. This example is for French.
|
||||
```
|
||||
{
|
||||
"name": "Français",
|
||||
"core-index/help": "Aide",
|
||||
(… more lines)
|
||||
}
|
||||
```
|
||||
|
||||
So the key `core-index/help` will render as `"Aide"` in French.
|
||||
|
||||
### Localization with a parameterized value ### {#localization-with-a-parameterized-value-}
|
||||
In this example, the name of the column (represented by `$1` in this example), will be substituted with the string of the name of the column.
|
||||
|
||||
`"core-facets/edit-facet-title": "Cliquez ici pour éditer le nom de la facette\nColonne : $1",`
|
||||
|
||||
### Localization with a singular/plural value ### {#localization-with-a-singularplural-value-}
|
||||
In this example, one of the parameter will have a different string depending if the value is 1 or another value.
|
||||
In this example, the string for page, the second parameter, `$2`, will have an « s » or not depending on the value of `$2`.
|
||||
|
||||
`"core-views/goto-page": "$1 de $2 {{plural:$2|page|pages}}"`
|
||||
|
||||
## Front End Coding {#front-end-coding}
|
||||
|
||||
The OpenRefine front end has been localized using the [Wikidata jquery.i18n library](https://github.com/OpenRefine/OpenRefine/pull/1285. The localized text is stored in a JSON dictionary on the server and retrieved with a new OpenRefine command.
|
||||
|
||||
### Adding a new string {#adding-a-new-string}
|
||||
|
||||
There should be no hard-coded language strings in the HTML or JSON used for the front end. If you need a new string, first check the existing strings to make sure there isn't an equivalent string, **in an equivalent context**, that you can reuse. Context is important because it can affect how the same literal English text is translated. This cuts down on the amount of text which needs to be translated.
|
||||
|
||||
Strings should be entire sentences or phrases and should include substitution variables for any parameters. Do not concatenate strings in either Java or Javascript (or implicitly by laying them out in a specific order). So, instead of `"You have " + count + " row(s)"` (or worse `count != 1 ? " rows" : " row"`), internationalize everything together so that it can be translated taking into account word ordering and plurals for different languages, ie `"You have $1 {{plural $1: row|rows}}"`, passing the parameter(s) into the `$.i18n` call.
|
||||
|
||||
If there's no string you can reuse, allocate an available key in the appropriate translation dictionary and add the default string, e.g.
|
||||
|
||||
```json
|
||||
...,
|
||||
"section/newkey": "new default string for this key",
|
||||
...
|
||||
```
|
||||
|
||||
and then set the text (or HTML) of your HTML element using i18n helper method. So given an HTML fragment like:
|
||||
```html
|
||||
<label id="new-element-id">[untranslated text would have appeared here before]</label>
|
||||
```
|
||||
we could set its text using:
|
||||
```
|
||||
$('#new-html-element-id').text($.i18n('section/newkey']));
|
||||
```
|
||||
or, if you need to embed HTML tags:
|
||||
```
|
||||
$('#new-html-element-id').html($.i18n('section/newkey']);
|
||||
```
|
||||
|
||||
### Adding a new language {#adding-a-new-language}
|
||||
|
||||
The language dictionaries are stored in the `langs` subdirectory for the module e.g.
|
||||
|
||||
* https://github.com/OpenRefine/OpenRefine/tree/master/main/webapp/modules/core/langs for the main interface
|
||||
* https://github.com/OpenRefine/OpenRefine/tree/master/extensions/gdata/module/langs for google spreadsheet connection
|
||||
* https://github.com/OpenRefine/OpenRefine/tree/master/extensions/database/module/langs for database via JDBC
|
||||
* https://github.com/OpenRefine/OpenRefine/tree/master/extensions/wikidata/module/langs for Wikidata
|
||||
|
||||
To add support for a new language, copy `translation-en.json` to `translation-<locale>.json` and have your translator translate all the value strings (ie right hand side).
|
||||
|
||||
#### Main interface {#main-interface}
|
||||
The translation is best done [with Weblate](https://hosted.weblate.org/engage/openrefine/?utm_source=widget). Files are periodically merged by the developer team.
|
||||
|
||||
Run the latest (hopefully cloned from github) version and check whether translated words fit to the layout. Not all items can be translated word by word, especially into non-Ìndo-European languages.
|
||||
|
||||
If you see any text which remains in English even when you have checked all items, please create bug report in the issue tracker so that the developers can fix it.
|
||||
|
||||
#### Extensions {#extensions}
|
||||
|
||||
Extensions can be translated via Weblate just like the core software.
|
||||
|
||||
The new extension for Wikidata contains lots of domain-specific concepts, with which you may not be familiar. The Wikidata may not have reconciliation service for your language. I recommend checking the glossary(https://www.wikidata.org/wiki/Wikidata:Glossary) to be consistent.
|
||||
|
||||
By default, the system tries to load the language file corresponding to the currently in-use browser language. To override this setting a new menu item ("Language Settings") has been added at the index page.
|
||||
To support a new language file, the developer should add a corresponding entry to the dropdown menu in this file: `/OpenRefine/main/webapp/modules/core/scripts/index/lang-settings-ui.html`. The entry should look like:
|
||||
```javascript
|
||||
<option value="<locale>">[Language Label]</option>
|
||||
```
|
||||
|
||||
## Server / Backend Coding {#server--backend-coding}
|
||||
|
||||
Currently no back end functions are translated, so things like error messages, undo history, etc may appear in English form. Rather than sending raw error text to the front end, it's better to send an error code which is translated into text on the front end. This allows for multiple languages to be supported.
|
||||
@ -197,7 +197,11 @@
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "version-3.5/technical-reference/translating"
|
||||
"id": "version-3.5/technical-reference/translating-ui"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "version-3.5/technical-reference/translating-docs"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user