Commit Graph

978 Commits

Author SHA1 Message Date
Tom Morris
348d82d131
Merge pull request #2725 from OpenRefine/issue-2724-wikidata-endpoint
Update URL of Wikidata reconciliation service
2020-06-15 17:20:05 -04:00
james-cui
04055153a1
add archive column (#2573)
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
2020-06-15 19:56:00 +02:00
Joanne Ong
d57d76f7df
Fix imprecise facet statistics in records mode (#2607)
* Fix bug in choice counts for records mode

* Add test for value grouper on records

* Refactor and comment code

* Count distinct instances of null/blank data

* Update test to check for blank data count in records

* Remove unnecessary import statement
2020-06-15 19:38:50 +02:00
Lisa Chandra
947356ddad
[FEAT]Adds new options for split (#2471)
* added options ui

* added definition for both separators

* added tests

* removed definitions from backend and added them to frontend

* added reverse order and handling for accented characters

* added tests for accented characters and reverse split

* fixed build errors

* unicode character ranges instead

* added examples
2020-06-15 19:30:18 +02:00
Antonin Delpeuch
1bb9e8a67e Update URL of Wikidata reconciliation service. Closes #2724 2020-06-15 00:35:10 +02:00
Tom Morris
bf1c890cc3
Unused imports and other minor cleanups (#2723)
* Two minor fixes

- prevent invalid index error on empty strings (shouldn't normally happen)
- update deprecated Apache Commons Lang method

* Remove unused imports
2020-06-14 21:18:02 +02:00
chuhao zeng
9b03ecae41
Convert illegal characters into legal ones. (#2431)
* Convert illegal characters into leagal ones.

* Test tab in key & value string

Also fix up test that depended on previous TAB
related error message and clean up logging

Co-authored-by: Tom Morris <tfmorris@gmail.com>
2020-06-14 09:47:58 +02:00
Tom Morris
18c18e587e
Replace Apache Ant with Commons Compress (#2691)
NOTE: Changes the public API where some of the old types were
embedded which means that any extensions that extend these
interfaces will have to be updated.

Fixes #2690.
2020-06-11 16:39:51 +02:00
Tom Morris
e6ed8e5d62
Save preferences JSON using UTF-8 encoding. Bulletproof prefs load. (#2657)
* Save preferences JSON using UTF-8 encoding. Bulletproof prefs load.

Fixes #2543. Fixes #2627.

Always use UTF-8 to write JSON because platform default encoding
might not be legal JSON (e.g. ISO 8859-1).

Also be more conservative about keeping backups if we fail to write.

* Handle case where backup prefs is better than more recent

* Recover from corrupted prefs with null starred list.

Fixes #2544. Replaces null with an empty list.

* Run tests with non-UTF-8 encoding

Make sure that we don't depend on UTF-8 being the default encoding
because it isn't true everywhere (e.g. Windows)

* Add test for non-ASCII chars in workspace.json

This depends on the default Java encoding being something
other than UTF-8 to test properly.
2020-06-06 10:00:01 +01:00
Antoine Beaubien
3ca08f6ff1
Changed cell.error to cell.errorMessage & added help data. (#2628)
* Changed cell.error to cell.errorMessage & added help data.

Changed cell.error to cell.errorMessage and added the informations into the internal help system.

* FR Text correction

* HU Fix text

3 instead of 2.
2020-05-23 14:05:25 +02:00
Lu Liu
e89eaf0ee2
support default project name and column name for cross() (#2518) 2020-05-22 09:39:57 +02:00
Tom Morris
557ffad920
Merge pull request #2586 from OpenRefine/issue-2510-type-boolean
Support "boolean" return for type() function.  Closes #2510
2020-05-18 17:24:47 -04:00
Antoine2711
0e86619d86 Fix the true.type() == "boolean"
Fix the true.type() == "boolean" instead of java.lang.Boolean.

Remove all the references to "error" result  in Type(). This will be addressed in:
@ToDo fix this with issue #2562
2020-05-18 17:23:43 -04:00
Antonin Delpeuch
d7d567439e Set version to 3.5-SNAPSHOT 2020-05-13 22:56:33 +02:00
Antonin Delpeuch
5597e1c942 Set version to 3.4-beta 2020-05-13 22:52:25 +02:00
Antonin Delpeuch
825e687b0b
Fix bug when both trim and autodetect are enabled in tabular parser. Closes #2584 (#2610) 2020-05-05 14:00:17 +02:00
Thad Guidry
15710ace17
reduce object creation during JSON serialization (#2576)
If a new {@code Double} instance is not required, this method
     * should generally be used in preference to the constructor
     * {@link #Double(double)}, as this method is likely to yield
     * significantly better space and time performance by caching
     * frequently requested values.
2020-05-05 10:07:54 +02:00
PJ Fanning
f047a88518
poi works better reading files directly (#2597) 2020-04-26 21:27:09 +02:00
PJ Fanning
ab64303cbb
allow xlsx files to have more columns (#2602) 2020-04-26 17:07:26 +02:00
PJ Fanning
1a0e187561
correct excel mime types (#2596)
* correct excel mime types

* address PR issue

* remove use of wildcard
2020-04-26 14:36:37 +02:00
PJ Fanning
88f7fb2852
Use SXSSFWorkbook in XlsExporter to improve memory usage when exporting xlsx files (#2594) 2020-04-26 12:26:05 +02:00
Thad Guidry
e5e2c8f665
remove Freebase AGENT_ID (#2575)
* remove unused imports

* remove unneeded Freebase AGENT_ID

In the past, Freebase editors used Google Refine for making edits to its database and the internal identifier was "/en/google_refine" which equated to a Software Application type with attached metadata and also had ownership privileges for certain Freebase Apps.  Since Freebase is no longer around, this identifier, only used by Freebase, can now be removed.  (This is not a User-Agent header string but was an internal identifier for the Freebase database which no longer exists)

* Revert "remove unused imports"

This reverts commit 9f6a276f36a54245016bd445680067d2c8862fcb.
2020-04-21 18:32:39 +02:00
Thad Guidry
009c587437
remove unused imports (#2574) 2020-04-21 15:51:01 +02:00
Lu Liu
bf84fc9cf1
use string representation for matching (#2571) 2020-04-20 09:07:09 +02:00
Ekta Mishra
05b6a7b2ae
Provides more intuitive representation for arrays in GREL (#2488)
Added test for same
closes #2040
2020-04-01 10:59:25 +02:00
chuhao zeng
1f0111eaed
Fix silent error in JSON/XML importers (#2414)
* Add error handler for parse error

* Add test for parsing json with incorrect strecture

* Enable localization from front-end

* Add methods to get localized error messages

* Update returned exception message

* Remove unused log and fix file diff issue

* Test auto build

* Refactor getOptions in newly created test

* Use new exception to unwrap original message

* Undo unexpected fix

* Remove unused lines

* Fix exception logic

* Fix typo
2020-03-27 09:41:49 +01:00
Albin Larsson
72966af5b6
remove Freebase reconciliation from Excel Importer (#2470) 2020-03-27 09:30:00 +01:00
Lu Liu
f2b06418da
Support lookup by numbers for GREL cross function (#2468)
* support int & long argument for cross function

* support any types of a cell value
2020-03-26 08:57:10 +01:00
chuhao zeng
70b4c6a6d0
Enable gzip compression (#2475)
* Enable gzip compression

* Add test for gzip parser
2020-03-26 08:42:55 +01:00
chuhao zeng
e484625adf
Fix: Data losses when importing multiple sheets from same Excell file (#2404)
* Fix loosing data when importing multiple sheets from same source Excell file

* Add test for importing multi sheets with different column size

* Fix space issues

* Restore old tests and implement new test cases for the new feature

* Restore unexpected delete

* Refactor fix

* Restore unexpected line delete

* Add new unit test for new feature
2020-03-23 22:41:23 +01:00
Thad Guidry
63bef81980
Remove unused variable in JSONUtilities (#2464) 2020-03-23 20:38:03 +01:00
Lu Liu
9ad3b1080f
Make cross() function work for all columns (#2456)
* fix #1950

* migrate from join to lookup

* reformat
2020-03-23 14:48:32 +01:00
Lisa Chandra
ef8ad85c3c
Adds trim whitespace option to separator based files (#2408)
* added trim ui to csv importer

* added trim functionality

* trimStrings handler only for strings

* added test for trimStrings option in csv/tsv files

* made trim option enabled by default
2020-03-21 10:38:43 +00:00
Albin Larsson
9745bfe374
consistent usage of Apache http status constants (#2432) 2020-03-18 06:40:52 +00:00
Lisa Chandra
a91691cb6b
[FIX] json/xml trim whitespace configuration option (#2415)
* trimStrings condition

* added test for trimString xml

* added trimStrings check for json
2020-03-15 16:04:01 +00:00
zengchu2
c90fd31daf
Add cell.error field for error messages (#2363)
* Add case for querying cell.error for error messages

* Add testing file

* Refactor test case for cell with error

* Reformat spaces
2020-03-10 10:14:15 +00:00
Chris Parker
93d34d781a Replaced some deprecated methods 2020-02-24 23:51:41 -06:00
Antonin Delpeuch
429f26c2ae Set version to 3.4-SNAPSHOT 2020-01-31 19:06:56 +01:00
Antonin Delpeuch
58b839b9c5 Set version to 3.3 2020-01-31 18:22:18 +01:00
Antonin Delpeuch
faece760f6 Set version to 3.3-SNAPSHOT 2020-01-08 20:56:51 +01:00
jamessspanggg
5afd93e2d1 Standardise 'edit' cell dialogue with 'toNumber()' behavior 2020-01-07 10:09:28 +08:00
Antonin Delpeuch
e62bb7ac0e Set version to 3.3-rc1 2020-01-06 13:30:39 +01:00
Antonin Delpeuch
904129d0f7 Fix other NPE in expression logging, for #2264 2020-01-06 06:30:56 +01:00
Antonin Delpeuch
14dd4c0112
Merge pull request #2264 from OpenRefine/issue-2086-expression-logging-npe
Fix NPE in expression logging.
2019-12-30 21:52:58 +01:00
Antonin Delpeuch
60089ab716
Merge pull request #2263 from OpenRefine/issue-2213-xlsx-export-url
More robust URI detection in tabular exporter.
2019-12-30 21:52:45 +01:00
Antonin Delpeuch
7593d5484d Add Hyperlink to cell in Excel importer, with fallback to String, for #2213 2019-12-25 22:24:58 +01:00
Antonin Delpeuch
08e175dc66 Fix NPE in expresion logging. Closes #2086. 2019-12-25 12:33:42 +01:00
Antonin Delpeuch
0bd6a0fbd7
Merge pull request #2198 from viniciusbds/master
Dealing with a possible null pointer dereference
2019-12-25 11:42:34 +01:00
Antonin Delpeuch
78853f8fb2 More robust URI detection in tabular exporter. Closes #2213. 2019-12-25 11:33:03 +01:00
Antonin Delpeuch
726395620b
Merge pull request #2202 from viniciusbds/patch-1
Update SqlCreateBuilder.java
2019-12-16 08:18:20 +01:00