s434730/RandomSec
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
4,900 Commits 6 Branches 0 Tags 1.3 GiB
480bf4fb85
Commit Graph

1073 Commits

Author SHA1 Message Date
Antonin Delpeuch
52989b78ce Fix typo 2018-05-23 07:39:04 +02:00
Antonin Delpeuch
d8f081b977 Remove DataExtensions from change serialization 2018-05-22 23:09:51 +02:00
Antonin Delpeuch
b01177c0d9
Merge pull request #1597 from OpenRefine/issue/1588-2 
unify the internal date type
 2018-05-18 22:03:14 +02:00
Owen Stephens
19a134d4ae Corrected spelling of Calender to Calendar 2018-05-17 10:01:04 +01:00
Jacky
52a2ec05e1
Merge pull request #1598 from ostephens/no-field-null 
No field null
 2018-05-16 16:01:05 -04:00
Jacky
a103e1fb90 merge the translation 2018-05-16 15:40:05 -04:00
Jacky
5adb3e455c fix the date format bug 2018-05-14 21:25:47 -04:00
Jacky
8fe27eff58 use yyyy-mm-dd formatter when converting date to string 2018-05-12 20:19:55 -04:00
Owen Stephens
13f2ebe9fc Always return null on attempt to get non-existent field, even from null 2018-05-10 16:37:38 -04:00
Jacky
c1132aa58a add nano support 2018-05-09 22:57:10 -04:00
Jacky
f58d963dbd Merge branch 'issue/1588' of https://github.com/OpenRefine/OpenRefine into issue/1588-2 2018-05-07 17:26:02 -04:00
Antonin Delpeuch
c0e39f507a
Merge pull request #1530 from OpenRefine/wikidata-extension 
Wikidata extension
 2018-05-06 08:46:53 +02:00
Antonin Delpeuch
8c2c4546d4
Merge pull request #1568 from OpenRefine/fix-mark-new-items 
Fix identifier space for cells reconciled to New
 2018-05-06 08:45:25 +02:00
Antonin Delpeuch
b8472cf6b8
Merge pull request #1573 from OpenRefine/issue472 
Fix FillDown and BlankDown operations on records.
 2018-05-06 08:45:01 +02:00
Jacky
19f98b7ea2 java 8 time API migrate 2018-05-05 11:41:40 -04:00
Jacky
7b857a4b24 implement nano date part 2018-05-02 22:02:35 -04:00
Jacky
34c46111ea undo the incidently push to master 2018-05-02 21:40:23 -04:00
Jacky
47d6f79e35 implement nano data part 2018-05-02 21:23:17 -04:00
Antonin Delpeuch
75b1863c1c Testing for variables which generate wikibase values 2018-04-27 09:17:32 +02:00
TonyO
0eb7b4082f added null and default value options 2018-04-26 20:50:46 -05:00
Antonin Delpeuch
c9224b2e06 Fix unused import 2018-04-22 11:01:14 +02:00
Antonin Delpeuch
1daa4d1312 Fix date parsing. 
Closes #1564.
 2018-04-22 10:49:22 +02:00
TonyO
c7c0d8884a check empty cells 2018-04-22 00:39:11 -05:00
Antonin Delpeuch
53c12ca2df Fix FillDown and BlankDown operations on records. 
Closes #472.
 2018-04-21 22:11:54 +02:00
Antonin Delpeuch
716a1606b4
Merge pull request #1558 from gobertm/master 
Contains function supports now regex as second argument
 2018-04-19 10:46:15 +02:00
Antonin Delpeuch
2b237433f9 Fix identifier space for cells reconciled to New 2018-04-15 01:20:25 +02:00
Jacky
de42f4d85a migrate from JRDF to JENA library 2018-04-08 22:38:23 -04:00
Antonin Delpeuch
eeef6055a0
Merge pull request #1525 from OpenRefine/google-api 
Google api
 2018-04-07 20:28:44 +02:00
Antonin Delpeuch
8943ef9390 Merge branch 'facet-by-null-and-empty-string' of https://github.com/joanneong/OpenRefine into joanneong-facet-by-null-and-empty-string 2018-04-04 17:37:51 +02:00
Maxime Gobert
a410639627 Contains support now regex as second argument 2018-03-28 17:25:11 +02:00
TonyO
000be9a783 removed unused code from Java classes 2018-03-26 17:33:26 -05:00
TonyO
3ffbcc9db7 remove whitespaces between column names options 2018-03-25 23:23:11 -05:00
TonyO
223d16db39 added ignore facet and trim column name options 2018-03-25 21:01:53 -05:00
TonyO
3834c27c8f SQL Export Code Commit 2018-03-24 22:41:07 -05:00
Jacky
3b7fc7350d refactoring the code the show the error message if there is any 2018-03-24 20:52:15 -04:00
Antonin Delpeuch
c536584991
Merge pull request #1536 from ostephens/coalesce 
Add coalesce function
 2018-03-18 13:06:52 +00:00
Owen Stephens
6e02baaa8a Coalesce must have at least two arguments 2018-03-17 16:21:05 +00:00
Joanne
64b01a2319 Add logic for new customized facets 2018-03-16 23:05:00 +08:00
Owen Stephens
74a6782ca0
Merge pull request #1524 from OpenRefine/issue/1508 
support project backward compatibility to old version of OpenRefine
 2018-03-07 09:36:13 +00:00
Owen Stephens
01b96a0155 Add coalesce function 2018-03-07 09:10:11 +00:00
Jacky
460fa611f1 sync with master 2018-03-06 19:28:28 -05:00
Jacky
77aad7eb05 changes based on the code review 2018-03-01 22:45:14 -05:00
Jacky
23ec54b78f support project backward compatibility to old version of OpenRefine 2018-02-28 20:16:29 -05:00
Jacky
54553da733
Merge pull request #1518 from OpenRefine/issue/1509 
Fix issue #1509
 2018-02-28 19:42:07 -05:00
Jacky
ab26bf4a27 fix issue #1517, limit is writen as int and should have been read as int 2018-02-25 17:22:34 -05:00
Jacky
036866837a add UT for issue #1509 2018-02-25 15:31:59 -05:00
Jacky
c758a6b635 exclude test-out 2018-02-25 12:00:40 -05:00
Jacky
32c5838a8d
Fix issue #1509 2018-02-23 11:15:56 -05:00
Antonin Delpeuch
3dcb3f1faf
Merge pull request #1515 from ostephens/conversions 
Conversions
 2018-02-22 09:49:49 +00:00
Owen Stephens
fad40041ca Use look up for function name 2018-02-21 23:20:47 +00:00
Owen Stephens
24f1923ff4 Return errors not nulls when: wrong number of arguments supplied; first argument zero length/empty string 2018-02-21 23:20:15 +00:00
Owen Stephens
5aaf4362b0 Return errors not nulls when: wrong number of arguments supplied; first argument zero length/empty string 2018-02-21 23:19:54 +00:00
Owen Stephens
d2687cc58a Treat null and empty string as different values when doing transforms 2018-02-18 23:19:28 +00:00
Joanne
41e8c0beee Update range function to return Integer arrays 2018-02-16 15:13:37 +00:00
Joanne
35c991d9c2 Update range function to follow Python's range function 2018-02-16 15:13:37 +00:00
Joanne
03d80598aa Update range function to include step ability 2018-02-16 15:13:37 +00:00
Joanne
bd2c4348fe Implement main logic for range function 2018-02-16 15:13:37 +00:00
Joanne
41f2a27a66 Register range function 2018-02-16 15:13:37 +00:00
Joanne
6cc2a1d353 Implement main logic behind range function 2018-02-16 15:13:37 +00:00
Owen Stephens
0d04a25cf1
Merge pull request #1450 from OpenRefine/issue1448 
Make Wikitext importer more robust
 2018-02-07 17:41:23 +00:00
Antonin Delpeuch
f5ff5565ff Cleanup for Codacy and better parsing of quote character 2018-02-03 16:37:15 +00:00
Thad Guidry
a191d13b14
adding missing StringUtils import 2018-02-03 07:26:47 -06:00
Thad Guidry
c83dbe3b23
Merge branch 'master' into stundzig-develop/1086-quotecharacter 2018-02-03 07:20:45 -06:00
Antonin Delpeuch
dc4815ba7b Fix initial options for the quote character 2018-02-03 09:00:53 +00:00
Antonin Delpeuch
c9395d564c Merge branch 'develop/1086-quotecharacter' of https://github.com/stundzig/OpenRefine into stundzig-develop/1086-quotecharacter 2018-02-03 07:56:30 +00:00
Jacky
c4b0ff6bea data package metadata (#1398) 
* fix the appbundle issue #1209

* fix #1162

allow the JRE 9

* fix the package declarations

* remove the _ from the method name

* use the explicit scoping

* remote extra ;

* fix issued from codacy

* fix issued from codacy

* add preferences link to the index page

* handle the empty user metadata

* fix 'last modified' sorting issue #1307

* prevent overflow of the table. issue #1306

* add isoDateParser to sort the date

* prevent overflow of the project index

* remove sorter arrow for action columns

* disable editing the internal metadata

* adjust the width of the table

* change MetaData to Metadata

* change the filed name from rowNumber to rowCount

* put back the incidently deleted gitignore

* add double quote to prevent word splitting

* UI improvement on metadata view and project list view

* remove the date field in metadata

* message notification of the free RAM. Issue #1295

* UI tuning for metadata view

* shorten the ISO date to locale date format

* Added translation using Weblate (Portuguese (Brazil))

* remove the rename link

* Ignore empty language files introduced by Weblate

* Add UI for Invert text filter

* Backend support for Inverting Text search facets

* Fix reset on text search facet

* More succinct return statements

* add tests for SetProjectMetadataCommand

* Tidying up for Codacy

* Added Tests for TextSearchFilter

* Corrections for Codacy

* More code tidy up

* let the browser auto fit the table cell when resizing/zooming

* fix import multiple excel with mulitple sheets issue #1328

* check if the project has the userMetadata

* fix the unit test
support multi files with multi tables for open office

* prevent the same key for user metadata

* replace _ with variable for exception

* fix the no-undef issue

* to adjust the width of transform dialog. issue #1332

* fix the row count refresh issue

* extract method

* move the log message

* cosmatic changes for codacy

* fix typo

* bump to version 2.8

* .gitignore is now working

* preview stage won't have the metadata populated, so protect NPE

* Update README.md

No more direct link to the last version tag, which will avoid having to think of updating the readme

* refacotring the ProjectMetadata class

* introduce the IMetadata interface

* create submodule of dataschema

* add back

* setup lib for dataschema; upgrade the apache lang to lang3

* replace escape* functions from apache lang3

* replace the ProjectMetadata with IMetadata interface

* add missing jars

* set the IMetadata a field of Project

* remove PreferenceStore out of Project model

* fix test SetProjectMetadataCommandTests by casting

* introdcue the AbstractMetadata

* introdcue the AbstractMetadata

* reorganize the metadata package

* allow have mulitiple metadata for a project

* support for mulitple metadata format

* remove jdk7 since 'table schema' java implmentation only support jdk8+

* set execute permission for script

* fix the Unit Test after Metadata refactoring

* restore the apache lang2.5 since jetty 6.1.22 depend on it

* add commons lang 2.5 jar

* git submodule add  https://github.com/frictionlessdata/datapackage-java

* remove the metadata parameter from the ProjectManager.registerProject method

* remove hashmap _projectsMetadata field from the ProjectManager and FileProjectManager

* init the Project.metadataMap

* fix Unit Test

* restore the ProjectMetaData map to ProjectManager

* put the ProjectMetaDta in place for ProjectManager and Project object

* check null of singleton instead of create a constructor just for test

* load the data package metadata

* importing data package

* importing data package

* encapsulate the Package class into DataPackageMetadata

* user _ to indicate the class fields

* introduce base URL in order to download the data files

* import data package UI and draft backend

* import data package UI

* fix typo

* download the data set pointed from metadata resource

* save and load the data package metadata

* avoid magic string

* package cleanup

* set the java_version to 1.8

* set the min jdk to 1.8

* add the 3rd party src in the build.xml

* skip the file selection page if only 1 DATA file

* add files structure for json editor

* seperate out the metadata file from the retrival file list

* rename the OKF_METADATA to DATAPACKAGE_METADATA

* clean up

* implement GetMetadateCommand class

* display the metadata in json format

* git submodule update --remote --merge

* adjust the setting after pulling from datapackage origin

* fix the failed UT DateExtensionTests.testFetchCounts due to new json jar json-20160810.jar will complain: JSONObject["float"] not a string.

* clean up the weird loop array syntax get complained

* remove the unused constant

* export in data package format

* interface cleanup

* fix UT

* edit the metadata

* add UT for SetMetadataCommand

* fix UT for SetMetadataCommand

* display the data package metadata link on the project index page

* update submodule

* log the exceptions

* Ajv does not work properly, use the back end validation instead

* enable the validation for jsoneditor

* first draft of the data validation

* create a map to hold the constraint and its handler

* rename

* support for minLength and maxLength from spec

* add validate command

* test the opeation instead of validate command

* rename the UT

* format the error message and push to the report

* fix row number

* add resource bundle for validator

* inject the code of the constrains

* make the StrSubstitutor works

* extract the type and format information

* add the customizedFormat to interface to allow format properly

* get rid of magic string

* take care of missing parts of the data package

* implement RequiredConstraint

* patch for number type

* add max/min constraints

* get the constrains directly from field

* implement the PatternConstraint

* suppress warning

* fix the broken UT when expecting 2 digits fraction

* handle the cast and type properly

* fix the missing resource files for data package when run from command line

* use the copy instead of copydir

* add script for appveyor

* update script for appveyor

* do recursive clone

* correct the git url

* fix clone path

* clone folder option does not work

* will put another PR for this. delete for now

* revert the interface method name

* lazy loading the project data

* disable the validate menu for now

* add UT

* assert UTs

* add UT

* fix #1386

* remove import

* test the thread

* Revert "test the thread"

This reverts commit 779214160055afe3ccdcc18c57b0c7c72e87c824.

* fix the URLCachingTest UT

* define the template data package

* tidy up the metadata interface

* check the http response code

* fix the package

* display user friendly message when URL path is not reachable

* populate the data package schema

* Delete hs_err_pid15194.log

* populate data package info

* add username  preference and it will be pulled as the creator of the metadata

* undo the project.updateColumnChange() and start to introduce the fields into the existing core model

* tightly integrate the data package metadata

* tightly integrate the data package metadata for project level

* remove the submodule

* move the edit botton

* clean up build

* load the new property

* load the project metadata

* fix issues from codacy

* remove unused fields and annotation

* check the http response code firstly

* import zipped data package

* allow without keywords

* process the zip data package from url

* merge the tags

* check store firstly

* remove the table schema src

* move the json schema files to schema dir

* add comment

* add comment

* remove git moduels

* add incidently deleted file

* fix typo

* remove SetMetadataCommand

* revert change

* merge from master
 2018-02-02 13:24:19 +00:00
Antonin Delpeuch
47f44b06c4 Make Codacy happy 2018-02-02 11:57:20 +00:00
Antonin Delpeuch
1f87a26f4c Include image file names like templates in Wikitable importer 2018-02-02 11:22:15 +00:00
Antonin Delpeuch
843641ca74 Parse headers just like other cells in Wikitable importer 
This makes the column names a bit less meaningful,
but this is necessary to handle cases where headers
are inserted anywhere in the table.
 2018-02-02 10:48:31 +00:00
Antonin Delpeuch
0426704938 Add helper createProjectWithColumns 2018-01-27 13:58:06 +00:00
Owen Stephens
af3aa44e6c Tidy up for codacy 2018-01-16 18:27:13 +00:00
Owen Stephens
7464e83766 Get and set any http headers submitted when using Add Column by Fetching URLs 2018-01-16 16:54:11 +00:00
Owen Stephens
4fb1867980 Option to add http headers in Add column by fetching URLs - support adding list of headers in backend, and exposing these in UI for user to edit and use 2018-01-16 16:54:11 +00:00
Jacky
9ab897cb72 remove copyright and put more clear description 2018-01-15 20:27:25 -05:00
Jacky
3c25647f05 add find function. issue 1296 2018-01-14 22:04:26 -05:00
Antonin Delpeuch
94b9d57528
Merge pull request #1397 from tiagofernandez/save-import-options 
Revert 278f24c1 and adapt 0001d9f7 (ref. PR #1387)
 2017-12-27 09:44:19 +01:00
Tiago Fernandez
00faf2a179 Revert 278f24c1 and adapt 0001d9f7 (ref. PR #1387) 2017-12-24 12:48:23 +01:00
Jacky
fb3620b3d9
Merge pull request #1388 from jackyq2015/issue/1386 
Fix Issue #1386
 2017-12-21 08:26:29 -05:00
jackyq2015
8b203a094d add the braces 2017-12-20 13:30:38 -05:00
Tiago Fernandez
0001d9f7b2 Test ImportingUtilities.createProjectMetadata() 2017-12-20 11:07:36 +01:00
jackyq2015
ad293a1419 add missing import 2017-12-19 15:00:21 -05:00
jackyq2015
81043e58e2 fix #1386 2017-12-19 12:54:26 -05:00
Tiago Fernandez
278f24c147 Append options metadata while creating project 2017-12-18 11:38:07 +01:00
Jacky
a55e0dab1c resolve the conflict 2017-12-15 22:07:07 -05:00
Jacky
d207c74f62 merge from master 2017-12-15 22:06:30 -05:00
Jacky
659f7bc27a fix for codacy 2017-12-15 21:46:09 -05:00
Antonin Delpeuch
3c5a9b0754
Merge pull request #1357 from xseris/master 
Tag system
 2017-12-16 01:55:46 +00:00
Jacky
93f60c848b fix a typo 2017-12-15 19:50:27 -05:00
Jacky
31a613f914 Revert "don't refresh row number each time" 
This reverts commit 880620b538.
 2017-12-15 17:38:34 -05:00
Jacky
2afdcbf546 fix issue https://github.com/jackyq2015/OpenRefine/issues/2 2017-12-15 16:37:38 -05:00
Jacky
0258dc1787 fix issue #1383. use the localDateTime for project info 2017-12-15 15:24:09 -05:00
Jacky
880620b538 don't refresh row number each time 2017-12-14 22:17:51 -05:00
xseris
ee8ed8f563 Refresh tag cell 2017-12-08 00:46:59 +01:00
xseris
b16941777e Modified tags in metadata dialog and moved column right to name 2017-12-07 23:45:36 +01:00
Owen Stephens
cc71f7f64f Remove qaResult option from backend 2017-12-04 09:45:24 +00:00
xseris
f21230804b Fixed Mockito tests 2017-12-01 17:00:45 +01:00
Owen Stephens
02f864121d Removed unused variable 2017-11-29 11:01:43 +00:00
Owen Stephens
0c97ad1b69 Use string instead of string buffer 2017-11-29 09:47:31 +00:00
Owen Stephens
316da5e917 Removed comment no longer needed 2017-11-29 01:00:19 +00:00
Owen Stephens
0b433fc5ec Updated copyright statement 2017-11-29 00:59:49 +00:00
Owen Stephens
1d1f3e67cd Added break statement for default case 2017-11-29 00:58:51 +00:00
Owen Stephens
21741ad09e Pass user friendly error message to UI on invalid regular expression 2017-11-28 16:22:17 +00:00
Owen Stephens
d64dee71dd Extended tests and code for user friendly regular expression syntax errors 2017-11-28 16:20:24 +00:00
xseris
d2648d8db8 Fix tests 2017-11-28 00:16:08 +01:00
xseris
cecbfb486b Remove delete all projects option 2017-11-24 20:29:09 +01:00
xseris
a3132eeeb3 implemented tag system 2017-11-24 20:19:49 +01:00
Owen Stephens
441d9f7f10 Initial tests and code for user friendly regular expression syntax errors 2017-11-24 18:24:16 +00:00
xseris
4f67c632d0 delete all projects 2017-11-23 17:36:46 +01:00
Qi Cui
ffbc6879ce preview stage won't have the metadata populated, so protect NPE 2017-11-18 23:02:06 -05:00
Qi Cui
ae9e0af8b9 bump to version 2.8 2017-11-18 14:15:10 -05:00
Thad Guidry
daca1836e9
Merge pull request #1342 from jackyq2015/feature/metadata 
fix the row count refresh issue.  We'll work on adding a Delete Metadata option in the next milestone.
 2017-11-18 08:30:32 -06:00
Jacky
4e79c0d410 fix typo 2017-11-17 22:55:16 -05:00
Jacky
428d944e4d move the log message 2017-11-17 16:58:30 -05:00
Jacky
c32f6329d2 extract method 2017-11-17 16:52:26 -05:00
Jacky
8187799f85 fix the row count refresh issue 2017-11-17 14:38:02 -05:00
Thad Guidry
43b6cb8001
Merge pull request #1333 from jackyq2015/feature/metadata 
check if the project has the userMetadata
 2017-11-16 22:17:14 -06:00
Jacky
8d2c8b521b
Merge pull request #1334 from jackyq2015/issue/1328 
fix import multiple excel with mulitple sheets issue #1328
 2017-11-16 21:30:24 -05:00
Jacky
e07b1859db replace _ with variable for exception 2017-11-16 15:55:14 -05:00
Jacky
1d2998f374 fix the unit test 
support multi files with multi tables for open office
 2017-11-16 10:29:16 -05:00
Antonin Delpeuch
78b7f01be5
Merge pull request #1327 from ostephens/invert-filter 
Invert text search facet/text filter
 2017-11-16 08:05:31 +00:00
Jacky
6f42df95e2 fix import multiple excel with mulitple sheets issue #1328 2017-11-15 22:58:47 -05:00
Owen Stephens
7ba110e31a More succinct return statements 2017-11-14 17:40:07 +00:00
Owen Stephens
135052d033 Backend support for Inverting Text search facets 2017-11-14 14:24:50 +00:00
Antonin Delpeuch
690ba49f53 Ignore empty language files introduced by Weblate 2017-11-14 09:40:20 +00:00
Jacky
a53467f2cb remove the date field in metadata 2017-11-09 22:29:08 -05:00
Jacky
428f7dabf8 change the filed name from rowNumber to rowCount 2017-11-08 20:40:22 -05:00
Jacky
31943f6c9f change MetaData to Metadata 2017-11-08 20:30:01 -05:00
Jacky
fc2f81046f adjust the width of the table 2017-11-08 17:49:49 -05:00
Jacky
c08b0f589a add isoDateParser to sort the date 2017-11-05 09:49:58 -05:00
Jacky
51691fc06d handle the empty user metadata 2017-11-04 19:07:09 -04:00
Jacky
d9995ba259 fix issued from codacy 2017-11-03 20:12:57 -04:00
Jacky
a204809f3d fix issued from codacy 2017-11-03 19:46:07 -04:00
Jacky
acf6460ea2 remote extra ; 2017-11-03 17:48:28 -04:00
Jacky
12c5839600 use the explicit scoping 2017-11-03 17:47:18 -04:00
Jacky
986881b10f remove the _ from the method name 2017-11-03 17:39:49 -04:00
Jacky
3a3083a1de Merge branch 'master' of https://github.com/OpenRefine/OpenRefine 2017-11-03 17:04:48 -04:00
Jacky
edc3df1ad1 populate the row number.\n sort the project index 2017-11-03 16:59:03 -04:00
Jacky
1777a6ecf3 edit the user metadata 2017-10-29 13:46:37 -04:00
Jacky
7c83746ee7 deal with empty string properly 2017-10-27 16:59:55 -04:00
Thad Guidry
d72a2de348 Revert "Extend cross() function to support multiple-value-cell-input" 2017-10-26 17:37:10 -05:00
Jacky
6a47482ea4 support metadata edit 2017-10-26 15:47:24 -04:00
Jacky
249fa4d8d5 support metadata edit 2017-10-26 15:45:58 -04:00
Thad Guidry
3d0e96a0ce Merge pull request #1290 from claussni/cross-func-split 
Extend cross() function to support multiple-value-cell-input
 2017-10-26 14:23:23 -05:00
Ralf Claussnitzer
0b107ec5e9 Add optional 4th parameter to cross() function 
The cross function now accepts a 4th parameter defining a regular
expression separator for splitting multi-value field values when joining
projects.

See https://github.com/OpenRefine/OpenRefine/issues/1204#issuecomment-326320954
 2017-10-26 19:50:02 +02:00
Ralf Claussnitzer
9aa168633f Allow comma separated multi-value source in cross() function 
Implements support for comma separated multiple-value keys for joining
another project using the cross() function.

See https://github.com/OpenRefine/OpenRefine/issues/1204#issuecomment-326320954
 2017-10-26 19:50:02 +02:00
Antonin Delpeuch
88b10a2917 Merge pull request #1278 from ostephens/cell-split-regex 
Cell split regex
 2017-10-25 11:04:33 +01:00
Owen Stephens
224210625d Remove automatic trim of split values 2017-10-24 08:28:37 +01:00
Owen Stephens
46c3ec100e Remove unused local variables and imports 2017-10-23 08:36:08 +01:00
Antonin Delpeuch
23b643426a Fix Codacy warnings in MultiValuedCellSplitOperation 2017-10-23 08:41:14 +02:00
Owen Stephens
cccf1e55c9 Update split multi-valued cells to support split by regex and split by lengths 2017-10-22 23:54:18 +01:00
Jacky
63c1714d0a add fields for metadata 2017-10-22 00:37:59 -04:00
Jacky
f1ab6b8cd6 Merge branch 'master' of https://github.com/OpenRefine/OpenRefine 2017-10-21 23:49:58 -04:00
Jacky
818e139b43 add the import options to metadata 2017-10-21 23:41:11 -04:00
Antonin Delpeuch
21f4d62474 Merge pull request #1275 from OpenRefine/wikitext-url-fix 
Forbid pipe characters in URL references to ease parsing.
 2017-10-20 16:41:00 +02:00
Antonin Delpeuch
e2a22a6994 Forbid pipe characters in URL references to ease parsing. 
This is a temporary fix before we do full Wikitext parsing inside references
(this needs a change upstream). See https://github.com/sweble/sweble-wikitext/issues/67 .
 2017-10-20 15:32:58 +01:00
Antonin Delpeuch
54acf10edf Change "topic" to "item" in the UI 2017-10-18 12:39:40 +01:00
Antonin Delpeuch
473b1b135d Merge pull request #1264 from OpenRefine/issue1262 
Update Jackson to 2.9.1
 2017-10-09 20:09:49 +02:00
Antonin Delpeuch
c9cc4fb262 Update Jackson to 2.9.1 
Closes #1262
 2017-10-09 17:38:09 +01:00
Owen Stephens
bb6b8378d3 Ensure _max is never less than _min 2017-10-09 17:13:43 +01:00
Antonin Delpeuch
1da3c00cb1 Perform ASCII normalization earlier in FingerprintKeyer. 
This closes #1256.
 2017-09-27 09:23:40 +01:00
Antonin Delpeuch
cfc0b95cd1 Fix string comparison in Wikitext exporter 2017-09-23 23:13:18 +01:00
Antonin Delpeuch
a1b2c9b683 Add support for references in Wikitable importer. 
Closes #1243.
 2017-09-23 22:54:43 +01:00
Antonin Delpeuch
49564e8905 Fix bug when an extra column starts in the middle of the table 2017-09-19 13:54:27 +01:00
Antonin Delpeuch
00f8e4fc6b Merge pull request #1237. 
Conflicts:
	.classpath
	main/webapp/modules/core/langs/translation-en.json
	main/webapp/modules/core/scripts/dialogs/extend-data-preview-dialog.js

Closes #363 and #56.
 2017-08-28 16:25:50 +01:00
Antonin Delpeuch
c66e609b1d Cleanup wikitext PR for Codacy 2017-08-26 21:50:02 +01:00
Antonin Delpeuch
0a00fd9318 Add option to include raw templates as cells 2017-08-25 14:28:30 +01:00
Antonin Delpeuch
554b75fa7b Fix parsing of newlines in cells 2017-08-17 19:18:50 +01:00
Antonin Delpeuch
7989aacc58 Cleanup for Codacy 2017-08-17 12:40:56 +01:00
Antonin Delpeuch
637e69db9d Better error reporting and testing for Wikitext import 2017-08-16 10:30:51 +01:00
Antonin Delpeuch
3dcda5a42c Add reconciliation config in wikitext import. 2017-08-16 00:05:40 +01:00
Antonin Delpeuch
86dc240335 Support reconciliation via sitelinks. 
Wikilinks are automatically reconciled at import time.

Related to #56.
 2017-08-15 20:17:34 +01:00
Antonin Delpeuch
aa4517ba58 Add support for colspan and rowspan in Wikitext 2017-08-15 11:28:43 +01:00
Antonin Delpeuch
73f7fdc036 Update TextFormatGuesser to support wikitext 2017-08-14 15:58:27 +01:00
Antonin Delpeuch
e168c900e8 Add support for table headers 2017-08-13 20:14:48 +01:00
Jacky
c3e04010b1 Merge branch 'master' into master 2017-08-13 14:09:56 -04:00
Antonin Delpeuch
b8a781d366 Add support for links (unreconciled for now) 2017-08-13 12:57:46 +01:00
Antonin Delpeuch
e6406f56ef Initial version of the wikitext importer 2017-08-13 11:26:59 +01:00
Antonin Delpeuch
dbb071da30 Merge branch 'default-to-english' of https://github.com/RBGKew/OpenRefine into RBGKew-default-to-english 2017-08-09 14:07:22 +01:00
Jacky
275dac976e fix #137 2017-08-07 21:53:35 -04:00
Antonin Delpeuch
66eac0fae9 Ensure null values are not cached in URL fetching operation. Closes #1219. 2017-08-01 13:05:29 +01:00
jackyq2015
53baa5a833 put the correct params description 2017-07-28 20:37:20 -04:00
jackyq2015
4950d29074 add backward compatility for cross function 2017-07-23 19:19:58 -04:00
Thad Guidry
7f92251ed1 Merge pull request #1210 from wetneb/extend 
Add data extension capabilities to the reconciliation API
 2017-07-17 18:01:37 -05:00
Antonin Delpeuch
84c06821ee Data extension tests 2017-07-16 11:47:12 +01:00
Antonin Delpeuch
05873f283d Integration of constraints with service-defined forms 2017-07-14 22:17:40 +01:00
Antonin Delpeuch
3eadefe613 Do not add reconciliation statistics on columns without types 2017-07-14 12:53:54 +01:00
Antonin Delpeuch
6501c235e8 Pass the identifier and schema spaces along to create better ReconCandidates 2017-07-14 12:30:39 +01:00
Antonin Delpeuch
cc991cab21 Add nicer spinning gif while preview is loading. 
Fix bug of multiple ColumnInfo being generated.
 2017-07-14 11:30:17 +01:00
Antonin Delpeuch
d99128c330 Retrieve types from the extend service 2017-07-06 21:15:37 +02:00
Antonin Delpeuch
ad3a174abd Starting to migrate data extension to standard reconciliation services 2017-07-04 23:14:19 +02:00
jackyq2015
1ee339cbbd cross function test suite. #1204 2017-06-28 08:12:36 -04:00
jackyq2015
f03be76475 Extend cross() function to take either a cell or a value #1204 2017-06-25 21:04:00 -04:00
Felix Lohmeier
2557cc5419 bugfix for new option autosave period 2017-06-24 22:42:49 +02:00
Felix Lohmeier
e54199a6f1 added options for initial java heap space and autosave period 2017-06-22 12:27:55 +02:00
Adi Eyal
09c00c6a19 Fixes #1181 2017-05-05 23:38:37 +02:00
Bob Harper
909df1b6a7 xor can also accept 2+ params, rewrite tests to be consistent 2017-04-27 11:20:48 +01:00
Bob Harper
ef4e039998 allow more than 2 AND and OR conditions 2017-04-26 20:51:58 +01:00
wangwenxiang
660df900d4 Fix bug: load wrong new value for RowStarChange 2017-03-15 12:54:01 +08:00
wangwenxiang
0314f49f36 Fix bug: load wrong new value for RowFlagChange 2017-03-15 10:39:33 +08:00
Jacky
912600f0bd Merge pull request #1178 from wetneb/url_caching 
Add caching in URL fetching
 2017-03-09 17:28:38 -05:00
Antonin Delpeuch
22124ac57e Add checkbox to disable caching 2017-03-09 00:21:34 +00:00
Antonin Delpeuch
32c232c2d6 Move to Guava's cache for ColumnAdditionByFetchingURLsOperation 2017-03-08 09:32:34 +00:00
Antonin Delpeuch
a9c4b0af16 Cache String, not URL, in ColumnAdditionByFetchingURLsOperation 2017-03-08 07:45:11 +00:00
Antonin Delpeuch
782a2f5b48 Add caching in URL fetching 2017-03-07 20:24:50 +00:00
Jacky
5aede573dc bump version to 2.7 2017-02-10 15:55:58 -05:00
Qi Cui
773151380e fix #1138. column transpose 2016-08-24 13:56:35 -04:00
Tom Morris
aa65bc5c18 Throw exception on error instead of logging to console 2016-05-17 15:10:09 -04:00
Tom Morris
6df822e5a6 Set ContentType to application/json 2016-05-17 15:10:09 -04:00
Tom Morris
5d45566455 Protect against NPE when content type is missing 2016-05-17 15:10:09 -04:00
Scott Wiedemann
16b0453b74 Update ToDate.java 
Updating SimpleDateFormat api doc url for ToDate function.
 2015-11-13 12:27:16 -07:00
Steffen Stundzig
7f5e58ef51 #1086 add support for quote character 2015-10-30 14:32:46 +01:00
Tom Morris
be7f880cbe Revert addition of synchronized methods 2015-10-16 19:33:15 -04:00
Tom Morris
e3858da843 Escape cell data for HTML - fixes #1049 2015-10-16 15:41:03 -04:00
Martin Magdinier
8b4a1d577a Merge pull request #1079 from RefinePro/issue-796 
fixed issue #796 Columnize by key/value columns creates empty lines
 2015-10-08 14:01:07 -04:00
jackyq2015
7a2a0eb52f fixed issue #796 Columnize by key/value columns creates empty lines 2015-09-29 20:12:05 -04:00
Tom Morris
48681e8877 Move assert where it belongs 2015-09-25 20:01:27 -04:00
Tom Morris
be936a86eb Clean up PR #1055 2015-09-25 19:01:16 -04:00
Tom Morris
de66afa512 Revert " Use new algorithm for levenshtein clustering" 2015-09-25 16:44:25 -04:00
Thad Guidry
175f4a5319 Merge pull request #1047 from lemmingapex/master 
Fixed #1046 Combine xls and xlsx formats by inspecting file header information in ExcelImporter
 2015-09-21 20:33:05 -05:00
Thad Guidry
94e219042e Merge pull request #1007 from lispc/master 
Use new algorithm for levenshtein clustering
 2015-09-21 20:23:45 -05:00
Thad Guidry
85ffce60d2 Merge pull request #1070 from RefinePro/issue-995 
fix issue #995
 2015-09-21 20:12:51 -05:00
jackyq2015
d671d7784b fix issue #995 2015-09-21 21:03:25 -04:00
magdmartin
ab56b73db9 Merge pull request #993 from RefinePro/OpenRefine-trunk 
prevent the multiple sorting
 2015-09-20 09:32:17 -04:00
magdmartin
b635f4e067 Merge pull request #1055 from RefinePro/issue-512 
fix issue #512 to save the file location as a table column
 2015-09-20 09:31:16 -04:00
magdmartin
ab6e2951e9 Merge pull request #1051 from RefinePro/issue-1015 
Issue 1015. add the meta utf-8
 2015-09-20 09:28:10 -04:00
jackyq2015
4e6f584cde fix issue #512 to save the file location as a table column 2015-08-27 15:13:20 -04:00
jackyq2015
dc7535c63e 1. take out of issue #1021 fix which was mistakenly put in 
2. fix the expected value for JUNIT
 2015-08-06 21:31:37 -04:00
Scott Wiedemann
5eab8893cc Fixed #1046 Combine xls and xlsx formats by inspecting file header information in ExcelImporter. 2015-07-30 16:19:26 -06:00
jackyq2015
819e1ba5c6 patch for issue #708. fix few hanging UIs when importing file 2015-07-18 10:27:35 -04:00
lispc
43e441a4d0 Use new algorithm for levenshtein clustering 2015-06-01 20:35:21 +08:00
Jacky
ca862970a4 prevent the multiple sorting 2015-05-01 15:04:51 -04:00
magdmartin
383f8c5e50 Changed GREL to *General Refine Expression Language* as agreed in 2013 when drafting *Using OpenRefine* 2015-04-21 10:35:52 -04:00
Matthew Blissett
5cdc6d7b5a Fallback to English language to avoid need to maintain 'default' translation files. 2015-02-10 12:33:08 +00:00
QI CUI
495dcd7bd5 use the LinkedHashMap instead of HashMap to make sure the retrive order 2015-01-30 15:03:20 -05:00
Tom Morris
83da996a36 Change to Java 5 loop syntax 2014-12-23 00:04:24 -05:00
Tom Morris
ddfaecb3e6 Merge pull request #914 from opendatatrentino/rev-masschange 
Fix wrong revert order in MassChange
 2014-12-22 23:50:31 -05:00
David Leoni
4d2b90ad60 added MassChangeTests 2014-12-22 12:23:49 +01:00
Tom Morris
ea723413cb Use StringUtils.toString() convenience method 2014-12-21 11:39:34 -05:00
Tom Morris
4eb6eb6eda Merge pull request #915 from opendatatrentino/fixNullCellToString 
Fixes Cell.toString failing on null value
 2014-12-21 11:13:34 -05:00
Matthew Blissett
f3e2b9622a Add charset=UTF-8 to HTTP Content-Type for reconciliation queries. 
Fixes problem where non-ASCII characters would be URL encoded as UTF-8, but interpreted according to the whims of the server.
 2014-11-28 10:45:22 +00:00
David Leoni
c3884c57f5 Fixes Cell.toString failing on null value 2014-11-27 18:45:01 +01:00
David Leoni
d29bf230b5 Fixes wrong revert order in MassChange 2014-11-27 18:12:54 +01:00
Thad Guidry
cdda1edcf0 Fixed issue with null cells after Fetch URL 
Some websites do not set the charset= properly and use enclosing quotes.  Tested and Verified.
 2014-08-13 21:39:30 -05:00
Tom Morris
536493c5d3 Fix AbstractMethodError 500 - fixes #589 2014-08-05 14:55:45 -04:00
Tom Morris
2fa9cf11c8 Merge pull request #859 from Arcadelia/Job-lastTouched-fix 
Initialized ImportingJob.lastTouched
 2014-07-03 10:36:48 -04:00
Tom Morris
655e0b0dc1 Wrap conditional statement in block 2014-07-03 10:35:24 -04:00
Tom Morris
b21cb56149 Merge pull request #852 from Arcadelia/Duplicate-job-id-fix 
Import job duplicate id fix
 2014-07-03 10:34:29 -04:00
Tom Morris
4333b1b2e7 Merge pull request #881 from zsxwing/simple-date-format-bug 
Put ISO8601_FORMAT into ThreadLocal to fix the concurrency issue
 2014-07-03 10:15:03 -04:00
Tom Morris
d106d61b25 Improve error messages - fixes #878 2014-05-30 01:47:22 -04:00
Tom Morris
5799c3d92b Synchronize access to processes list - fixes #862 2014-05-30 01:47:21 -04:00
zsxwing
4ee8e079c9 Put ISO8601_FORMAT into ThreadLocal to fix the concurrency issue 2014-05-30 11:45:28 +08:00
Tom Morris
a4d03968a5 Merge pull request #867 from abhillman/exceloutput255bugfix 
Report error to user when attempting to export >255 columns, rather than generic 500 ISE
 2014-04-20 23:43:19 -04:00
Aryeh Hillman
2bf35e5f0d Fix when exporting to excel files 
When exporting to excel, there cannot be more than 255 columns.
If there are more columns than that, we write "ERROR: TOO MANY
COLUMNS" to the 255th column. Formerly, OpenRefine reported
a 500 Server error.
 2014-04-12 16:41:54 -07:00
Frank Wennerdahl
8c02a13429 Initialized ImportingJob.lastTouched 
Prevents the CleaningTimerTask from disposing newly created
ImportingJobs which have not yet been touched.
 2014-02-19 16:02:45 +01:00
Frank Wennerdahl
a0d4eb0058 Job id duplicate fix 
Changed how job id's are created to avoid the same id to be assigned to
two concurrent jobs.
 2014-02-05 12:21:50 +01:00
Frank Wennerdahl
6dedae37a1 Fixed too frequent job cleanups 
The ImportingManager cleans up jobs that has not been touched in 60ms.
According to comment this should be 60 minutes but was changed in
4529310237.
 2014-02-05 11:07:41 +01:00
Tom Morris
bc801546cc Remove references to obsolete splitIntoColumns option 2013-09-18 18:44:58 -04:00
Tom Morris
4f2ebed676 Make localization language list dynamic - fixes #807 
- refactor LoadLanguageCommand so language loading can be reused
- add GetLanguagesCommand for the server
- change GUI to fetch language list and update selection list with it
 2013-09-18 13:16:24 -04:00
Tom Morris
1261734f15 Partial solution for #816 plus improved conversion test coverage 2013-09-18 11:14:48 -04:00
Tom Morris
d84f897ae0 Improve help message to specify an integer is returned 2013-09-18 11:12:34 -04:00
Tom Morris
f344e3da1c Return "null" for toString(null) - fixes #783 
- also fixed grammar in error message
 2013-09-18 10:20:17 -04:00
Tom Morris
daed3bd90c Move MARC->XML conversion to earlier in process - issue #794 
- functional now, but probably not good enough to release yet
 2013-09-17 19:19:50 -04:00
Tom Morris
6bd6a5934b Start wiring up MARC importer - issue #794 2013-09-17 17:17:23 -04:00
Tom Morris
cce480ff38 Fix implementation for #466 to handle default empty string 2013-09-04 18:59:13 -04:00
Tom Morris
889245fdf4 Make the number of reconciliation results configurable - closes #466 2013-09-04 18:07:12 -04:00
Thad Guidry
f2c4e3ab48 Added ability to extract MILLISECOND to datePart (milliseconds,ms,S) 2013-08-30 09:09:54 -05:00
Tom Morris
c68c1bb2b1 Upgrade to Clojure 1.5.1 & switch to clojure-slim JAR - #792 2013-08-26 19:40:37 -04:00
Tom Morris
62b8c476f1 Use Java's built-in Number formatter instead of ICU4J which is 
massive - #792
 2013-08-26 15:47:12 -04:00
Tom Morris
4529310237 Switch from TimerTask to ScheduledExecutorService for more robustness 2013-08-18 11:31:03 -04:00
Tom Morris
e93bfa798e Use iterator when removing to avoid ConcurrentModificationException - 
fixes #652
 2013-08-17 13:45:22 -04:00
Tom Morris
3315136681 Allow reinitializatoin of ProjectManager singleton - fixes #787 2013-08-17 12:47:57 -04:00
Tom Morris
25f02dd9b9 Fix Java 6 incompatibility 2013-08-15 15:57:24 -04:00
Tom Morris
fa072df85c Add locale support to toDate() - fixes #729 2013-08-15 15:19:01 -04:00
Tom Morris
ab42df6ea3 Merge pull request #658 from Arcadelia/CSV_Multi-char-separator_support 
Support for multi-char-separators in CSV
 2013-08-14 07:29:45 -07:00
Tom Morris
37d8abc114 Minor improvement to recon error handling 2013-08-10 18:03:06 -04:00
Tom Morris
1d8784e059 Make workspace saving and loading more robust - fixes #528 
- don't overwrite old files if we get an error writing new ones
- don't write unchanged data
- keep backup files around until next write rather than deleting
immediately
- attempt to recreate missing metadata as best as possible
 2013-08-09 19:53:53 -04:00
Tom Morris
579d71b7eb Switch back to NUL character for quote now that OpenCSV handles it - 
fixes #653
 2013-08-07 17:07:17 -04:00
Tom Morris
7b5b549113 More project saving changes for #528 
- reduce project retention in memory from 1 hr to 15 min.
- free all unmodified projects if we get an error on save (we could be
running low on memory)
- make sure exceptions propagate up to where they can be usefully
handled
 2013-08-05 14:13:56 -04:00
Tom Morris
190a031a8a Comments only. No code changes. 2013-08-05 14:11:06 -04:00
Tom Morris
3500f20e47 Save all modified projects before importing new one - hopefully helps 
#528
 2013-08-05 14:10:26 -04:00
Tom Morris
57f5e9873d Add Javadoc. No code changes. 2013-08-05 13:08:30 -04:00
Tom Morris
c3cab0524a Narrow exceptions thrown and let them propagate up so we know 
workspace file isn't valid - first step for #528
 2013-08-05 13:08:02 -04:00
Tom Morris
a7273625d7 Add support for Basic Authentication over HTTPS - addresses #217 2013-08-02 19:15:24 -04:00
Tom Morris
4f7da9d18e Switch to Apache HTTP client for downloads - fixes #748 2013-08-02 18:13:41 -04:00
Tom Morris
d7531bbbd8 Handle quoted fields with embedded new lines. Sort separators by score 
rather than just standard deviation
 2013-08-02 17:59:09 -04:00
Tom Morris
f4ff227340 Clean up localization - fixes #760, modifies pull request #755 
- make all file loading relative to module base
- move core language files into appropriate place
- eliminate all SetLanguage commands and use SetPreference instead
- eliminate all LoadLanguage commands except for core's
- fix duplicate keys in JSON language files
- remove BOM from JSON language files

OPEN - task 760: Translations not being loaded from built kit 
http://github.com/OpenRefine/OpenRefine/issues/issue/760
 2013-07-31 00:31:31 -04:00
Tom Morris
9450d483ce Fix up line endings 2013-07-29 15:49:20 -04:00
Tom Morris
3003c1a709 Make importers more robust to preview errors when someone selects the 
wrong importer/parser
 2013-07-27 13:35:12 -04:00
Tom Morris
57ca70132c Turn all import conversions off by default - fixes #478 2013-07-27 13:32:26 -04:00
Tom Morris
5123dad6a8 More conservative approach for locking of jobs table 2013-07-26 18:51:08 -04:00
Tom Morris
0dc14af1aa Fix bug in refactoring of ImportingJob from commit 
1e5f89e84c
 2013-07-26 18:50:03 -04:00
Tom Morris
46a1e198d8 Recompute max cell index when rebuiling maps in ColumnModel - fixes #406 2013-07-26 18:48:20 -04:00
Tom Morris
7edc550618 Give a reasonable error message on Excel 95 import failure - fixes #564 2013-07-26 16:24:56 -04:00
Tom Morris
dc4d04c132 Allow arrays containing null in Filter & ForEach - fixes #741 2013-07-26 15:20:44 -04:00
Tom Morris
1e5f89e84c Centralize handling of import job config object & synchronize to allow 
multiple accessors
 2013-07-25 15:41:08 -04:00
Tom Morris
dc206e1889 Switch to ConcurrentHashMap for jobs table to allow multiple accessors 2013-07-25 15:36:54 -04:00
Tom Morris
0ff2d7ed9f Simplify implementation from pull request #728 2013-07-25 13:45:44 -04:00
Tom Morris
6dd4b8ea23 Add tests for boolean functions and tighten up error handling 2013-07-25 13:45:04 -04:00
Tom Morris
2c2c0d3d68 Merge pull request #728 from jmcastagnetto/master 
Implements Xor operation
 2013-07-25 10:00:11 -07:00
Blakko
6e90bc41f6 Merge remote-tracking branch 'origin/master' into internationalization 
Conflicts:
	extensions/freebase/module/scripts/dialogs/schema-alignment/schema-alignment-dialog.html
	main/webapp/modules/core/index.vt
	main/webapp/modules/core/project.vt
	main/webapp/modules/core/scripts/project/browsing-engine.js
	main/webapp/modules/core/scripts/project/history-panel.html
 2013-07-25 11:07:59 +02:00
Blakko
e6e6c8c002 Added a "Language Settings" menu at index 
Now the language manually set has priority over the browser lang
Update translations
 2013-07-12 11:12:33 +02:00
Tom Morris
92e4427c39 Adding a TODO 2013-07-10 15:13:22 -04:00
Tom Morris
32773122c4 Fix CollationKey creation - fixes #753 2013-07-10 15:12:49 -04:00
Blakko
552b0bf94b Internationalization of the index part (create/open/update) of refine 2013-07-02 13:40:50 +02:00
Tom Morris
5b6bc888f7 Fix template escape processing. Fixes #752. 2013-06-30 12:21:26 -04:00
Tom Morris
a3b4b45e4e Support non-string types in facetCount() - fixes #591 2013-06-23 12:04:48 -04:00
Tom Morris
51c1bc4a2f Refactor default toString with date support into separate utility 2013-06-23 12:02:13 -04:00
Tom Morris
c961bb64de Flush all column caches on row removals/changes. Fixes issue 567. 2013-06-22 18:44:26 -04:00
Tom Morris
fd58bd3327 Move documentation to Javadoc where it's visible 2013-06-22 16:27:18 -04:00
Tom Morris
6e88d068ee Throw a narrower exception 2013-06-22 16:26:45 -04:00
Jesus M. Castagnetto
0795bd8422 resolved .gitignore conflict 2013-06-19 12:10:32 -05:00
Jesus M. Castagnetto
b09bb4463e fix error in index caught by thadguidry 2013-06-19 11:21:26 -05:00
Tom Morris
b91fc8a2b1 Use CollationKeys when sorting text. Fixes issue 738 2013-06-17 15:51:29 -04:00
Tom Morris
067fcacec7 Clean up to pass tests: 
- don't include TAB in control characters which get stripped so we can
use it for splitting
- remove trailing space from normalize strings
 2013-05-31 17:06:03 -04:00
Tom Morris
000c0a38a8 Compute delay from request issue, not response return. Fixes #721 2013-05-26 10:13:16 -04:00
Tom Morris
4a5d3d4662 Convert dates to ISO 8601 for reconciliation. Fixes #688. 2013-05-26 10:08:55 -04:00
Tom Morris
7615db97cf Add Javadoc clean up variable naming. No functional change. 2013-05-26 10:07:37 -04:00
Tom Morris
36dd95c263 Add TODO for record mode operation 2013-05-26 07:54:33 -04:00
Tom Morris
567da6aa9f Normalize line endings 
Add .gitattributes & do one-time normalization of line endings
 2013-03-23 18:46:20 -04:00
Tom Morris
6a91b5d75b Use InputStream instead of Reader for JSON import - fixes #698 2013-03-23 18:36:05 -04:00
Tom Morris
6b3592982e Remove O(n^2) issue in tree importers - fixes #699 
- Add sparse/based list implementation for ImportRecord
 2013-03-23 12:02:51 -04:00
Tom Morris
f78dfadcf3 Clean up tree import utilities for #699 
- lazy allocate objects
- conditionalize logging to prevent calls to StringBuilder & toString()

These are secondary issues, but still worth cleaning up.
 2013-03-23 11:56:58 -04:00
Tom Morris
0a2ba1b1ae Switch from LinkedList to ArrayList 
Just a simple list.  No need for extra overhead..
 2013-03-23 08:16:23 -04:00
Tom Morris
bfa7c34d17 Merge pull request #659 - closes #659 2013-03-18 21:24:01 -04:00
Tom Morris
31cffa1181 Merge remote-tracking branch 'upstream/master' 2013-03-18 21:16:55 -04:00
Tom Morris
8a61cf731b Merge pull request #664 from Arcadelia/Preserve_Quotes 
Quotes should not be removed from values
 2013-03-18 18:12:51 -07:00
Tom Morris
fe943fe3ea Flag English specific stopwords for cleanupp 2013-03-18 20:20:46 -04:00
Tom Morris
7b9f6836e1 Update key & id recon to new Freebase APIs - part of #696 2013-03-12 16:50:23 -04:00
Tom Morris
7578d3375f Add logger and logging 
- fix exception printing that goes nowhere
- make logger available for subclasses to use
 2013-03-11 13:14:20 -04:00
Tom Morris
a2a8f4af2e Patch applied - closed #315 2013-03-06 21:45:54 -05:00
Tom Morris
d8d82bf8b7 Clean up a couple more format guessing issues left over from #685 2013-03-06 20:39:39 -05:00
Tom Morris
369bfffb2f Don't guess field widths unless we have at least 3 lines 
- Investigation of #685 showed that single line files were being guessed
as fixed field width
 2013-03-04 17:47:06 -05:00
Tom Morris
6b676f7513 Handle MIME media types which have charset param - fixes #685 2013-03-04 17:45:34 -05:00
Tom Morris
10bd7e3b75 Make upper bound of time facet inclusive - fixes issue #648 2013-03-03 16:06:20 -05:00
Tom Morris
eba03fc69e Protect joins map with mutex - fixes issue #652 2013-03-03 09:36:43 -05:00
Tom Morris
7b3379afc7 fix range check in getFields - fixes issue 687 2013-02-26 16:35:21 -05:00
Tom Morris
389e762251 Merge remote-tracking branch 'upstream/master' 2013-02-26 00:01:06 -05:00
Tom Morris
95e13eac50 Improve recon error handling 2013-02-26 00:00:03 -05:00
Tom Morris
50888c6f2e Merge pull request #666 from Arcadelia/Temp-file_removal 
Fixed removal of upload temp files
 2013-02-11 15:11:24 -08:00
Tom Morris
1033ce973e TODO about memory usage 2013-02-03 15:56:54 -05:00
Jesus M. Castagnetto
71f3196048 added comment on implementation 2013-02-01 23:45:43 -05:00
Jesus M. Castagnetto
36d2c4ac44 Added full text of BSD 2-clause 2013-02-01 23:44:35 -05:00
Jesus M. Castagnetto
df450b20f7 Registering new XOR command 2013-02-01 22:42:01 -05:00
Jesus Castagnetto
fec35a8bc6 Update main/src/com/google/refine/expr/functions/booleans/Xor.java 2013-02-01 21:07:42 -05:00
Jesus Castagnetto
ebec459cfd indentation change 2013-02-01 21:00:36 -05:00
Jesus Castagnetto
473e2f367f Implementing Xor operation 2013-02-01 17:59:16 -08:00
Tom Morris
c0347225b8 Switch escape character from NUL to DEL in hopes that it's rarer. 2013-02-01 17:12:07 -05:00
Frank Wennerdahl
2c59a0059f Fixed removal of upload temp files 
Fixed an issue with an unclosed stream preventing upload temp files from
being removed after use. Also removed the use of FileCleaningTracker and
instead added manual removal of all tempfiles. By doing this the reaper
threads in FileCleaningTracker are avoided and files are removed
directly after use.
 2013-01-24 09:59:09 +01:00
Frank Wennerdahl
64cf62e081 Fixed history and header update in IE 
Due to Internet Explorer caching GET requests the Undo/Redo list and
column headers were not updated, leaving essential parts of the user
interface crippled even if Google Frame is installed. Adding
Cache-Control headers to the responses fixes this.
 2013-01-24 09:39:12 +01:00
Frank Wennerdahl
1f7ab046c7 Quotes should not be removed from values 
Leading quotation marks should not be removed from values. If they have
been left by the importing parser they should be considered part of the
value.
 2013-01-24 09:04:17 +01:00
Frank Wennerdahl
ebdc40ad71 Added CSV quote options 
Added two additional CSV options, one for parsing and one for export.

Specifying strict quotes when parsing will ignore all data not quoted.
Specifying quote all when exporting will enclose all values in quotes.

No front-end changes made, just added the support for the options in the
requests.
 2013-01-21 08:21:16 +01:00
Frank Wennerdahl
f837643f1e Support for multi-char-separators in CSV 
This change requires that the following patch is applied to OpenCSV:

http://sourceforge.net/tracker/index.php?func=detail&aid=3599477&group_id=148905&atid=773543
 2013-01-18 16:28:27 +01:00
Tom Morris
33aa1132d7 Clarify wording/naming of blank rows export option - fixes issue #651 
- clarify that it refers to all non-null cells
- rename variables without compatibility constraints to match actual
function
 2013-01-14 16:36:09 -05:00
Tom Morris
0bd2104a16 Issue 630: Change branding from Google Refine to OpenRefine 
** The first native Github commit (ie not one converted from SVN **
Change Google Refine to OpenRefine or just Refine.  
Change icon filenames and add some placeholder icons
 2012-10-18 19:40:31 -04:00
Tom Morris
068e0916a2 FIXED - task 587: Correct initialization of the temporary directory - patch from the Wikier project 
http://code.google.com/p/google-refine/issues/detail?id=587
https://bitbucket.org/wikier/google-refine/changeset/f3dbdb16a320#chg-main/src/com/google/refine/RefineServlet.java

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2583 7d457c2a-affb-35e4-300a-418c747d4874
 2012-10-13 15:58:44 +00:00
Tom Morris
4d48741ce0 FIXED - task 574: create safe sheet names for Excel export - patch from jd@tekii.com.ar 
http://code.google.com/p/google-refine/issues/detail?id=574

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2582 7d457c2a-affb-35e4-300a-418c747d4874
 2012-10-12 23:05:17 +00:00
Tom Morris
ca2e959957 FIXED - task 529: Add support for key/value transpose with only two columns as well as repeating key fields in a single record. 
http://code.google.com/p/google-refine/issues/detail?id=529

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2574 7d457c2a-affb-35e4-300a-418c747d4874
 2012-10-05 23:31:25 +00:00
Tom Morris
ffe674729c Just a little Javadoc. No functional changes. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2573 7d457c2a-affb-35e4-300a-418c747d4874
 2012-10-05 21:10:32 +00:00
Tom Morris
2c52a00f55 Fixed - issue 544,600,618: Clean up handling of compressed files & archives with multi-segment paths 
http://code.google.com/p/google-refine/issues/detail?id=600


git-svn-id: http://google-refine.googlecode.com/svn/trunk@2569 7d457c2a-affb-35e4-300a-418c747d4874
 2012-09-22 18:08:56 +00:00
Tom Morris
748e205ae8 FIXED - task 616: Support bzip2 decompression on import 
http://code.google.com/p/google-refine/issues/detail?id=616

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2568 7d457c2a-affb-35e4-300a-418c747d4874
 2012-09-22 16:00:42 +00:00
Tom Morris
27e3c0c8dc FIXED - task 614: Use same instance of OAuthProvider in OAuth dance. Patch supplied by sdeo@google.com 
http://code.google.com/p/google-refine/issues/detail?id=614

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2566 7d457c2a-affb-35e4-300a-418c747d4874
 2012-09-19 23:16:29 +00:00
Tom Morris
b3f5fada95 FIXED - task 578 & 596: Clean up JSON importer 
http://code.google.com/p/google-refine/issues/detail?id=578
http://code.google.com/p/google-refine/issues/detail?id=596

Extend tree parser framework to allow any Serializable instead of just Strings. Use this in JSON importer to: Import keywords null, true, false; Import empty strings and don't trim whitespace from strings on import;  Import numbers directly instead of importing them as text and then parsing them ourselves. Add tests to verify all this stuff

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2543 7d457c2a-affb-35e4-300a-418c747d4874
 2012-09-08 01:20:25 +00:00
Tom Morris
93d6e176d6 Task 478: Default "guess datatypes" to False so importers which don't specify it (e.g. gData & Excel) aren't effected 
http://code.google.com/p/google-refine/issues/detail?id=478

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2541 7d457c2a-affb-35e4-300a-418c747d4874
 2012-09-07 21:17:34 +00:00
Tom Morris
83dce305cb FIXED - task 432: cross() failing - flush join cache table when column changes 
http://code.google.com/p/google-refine/issues/detail?id=432

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2539 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-30 16:31:22 +00:00
Tom Morris
9b54a8f29e FIXED - task 559: Deadlock between autosave thread and history code 
http://code.google.com/p/google-refine/issues/detail?id=559

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2538 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-30 16:22:28 +00:00
Stefano Mazzocchi
ba89daec1c make oauth against freebase work again in chrome 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2537 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-23 20:58:08 +00:00
Tom Morris
12a61b6ec6 task 603: range check column move commands 
http://code.google.com/p/google-refine/issues/detail?id=603

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2534 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-18 22:01:23 +00:00
Tom Morris
202018fac4 Add Javadoc. No code changes. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2533 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-18 22:00:41 +00:00
Tom Morris
4bb6c43982 task 604: add Guava to main project so that we're not dependent on an extension 
http://code.google.com/p/google-refine/issues/detail?id=604

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2531 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-15 13:33:17 +00:00
Tom Morris
1e043dcc94 FIXED - task 604: The common transform “Trim leading and trailing whitespace” doesn’t trim non-breaking spaces 
http://code.google.com/p/google-refine/issues/detail?id=604

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2529 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-14 23:00:13 +00:00
Tom Morris
f29f77e8f8 STARTED - task 604: The common transform “Trim leading and trailing whitespace” doesn’t trim non-breaking spaces 
http://code.google.com/p/google-refine/issues/detail?id=604

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2528 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-14 20:09:49 +00:00
Tom Morris
4bf212c03d FIXED - task 154: Can't import RDF/XML Data 
http://code.google.com/p/google-refine/issues/detail?id=154

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2526 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-05 16:31:41 +00:00
Tom Morris
5881addac8 Throw an exception if unsupported verb is used 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2525 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-05 15:36:23 +00:00
Tom Morris
b2ae74d23f FIXED - task 586: Only one parse date format is attempted from list in toDate(format1,format2) 
http://code.google.com/p/google-refine/issues/detail?id=586

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2520 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-03 18:01:01 +00:00
Tom Morris
4319314675 FIXED - task 594: Date diff function doesn't work for two Calendar objects 
http://code.google.com/p/google-refine/issues/detail?id=594

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2519 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-02 21:41:19 +00:00
Tom Morris
efa58630cf Add constructor that takes a Throwable to eliminate redundant code from callers. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2518 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-02 21:38:00 +00:00
Stefano Mazzocchi
2cb31b8b29 fixing oauth problems with redirection for the Freebase API 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2516 7d457c2a-affb-35e4-300a-418c747d4874
 2012-08-01 21:46:53 +00:00
David Huynh
4cfb921082 Added getStringKey() method for when it is difficult to generate integer keys that don't collide 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2515 7d457c2a-affb-35e4-300a-418c747d4874
 2012-07-19 00:25:41 +00:00
Stefano Mazzocchi
6e41f4ad91 make the latest eclipse happy (it triggers a warning) 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2513 7d457c2a-affb-35e4-300a-418c747d4874
 2012-07-12 01:55:11 +00:00
Stefano Mazzocchi
bccea8cebe we could be leaking file descriptors here 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2506 7d457c2a-affb-35e4-300a-418c747d4874
 2012-06-30 07:05:08 +00:00
Stefano Mazzocchi
f84dcff900 moving oauth authorize and deauthrorize into the core module because they are reusable across extensions 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2505 7d457c2a-affb-35e4-300a-418c747d4874
 2012-06-29 19:39:42 +00:00
Tom Morris
8872c1b0a1 Keep track of when we have unsaved preference changes 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2502 7d457c2a-affb-35e4-300a-418c747d4874
 2012-06-02 21:06:46 +00:00
Tom Morris
a0812c5751 Be slightly more tolerant of weird spreadsheet data 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2501 7d457c2a-affb-35e4-300a-418c747d4874
 2012-06-02 21:00:30 +00:00
Tom Morris
c47b1e0ab7 Mark project as modified when metadata is changed 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2491 7d457c2a-affb-35e4-300a-418c747d4874
 2012-04-14 14:10:11 +00:00
Tom Morris
8d22ede1f8 Issue 554 - rank formats *before* serializing them. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2482 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-16 20:21:57 +00:00
Tom Morris
b3f8ce83c1 Issue 553 - Make sure we have a usable filename when importing from a URL 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2481 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-16 20:16:18 +00:00
Tom Morris
51c586bc2c Issue 543 - Handle HTTP responses with Content-Encoding of gzip 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2480 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-16 20:12:10 +00:00
Tom Morris
a8cb23ca51 Issue 544 - preserve directory path after decompressing file 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2479 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-16 20:06:54 +00:00
Tom Morris
e97e7523b2 Issue 548 - Convert non-strings to strings before escaping 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2463 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-14 03:06:11 +00:00
Tom Morris
18b780bebe Issue 517 - Fix combin() function to a) increase upper limit and b) keep it from continually recomputing the same values in recursion 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2459 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 22:53:21 +00:00
Tom Morris
28ff2295fd Issue 490 - Handle separator guessing for CSVs with quoted fields containing commas 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2458 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 15:53:55 +00:00
Tom Morris
9a680e8307 Switch to class name for logging, per convention 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2457 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 14:53:27 +00:00
Tom Morris
ddd3680128 Add a TODO for recon failure retries on HTTP 500s - no functional changes 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2455 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 14:45:53 +00:00
Tom Morris
5a962b1768 Issue 534 - Attempt to recover recon links which have become corrupted 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2454 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 00:37:29 +00:00
Tom Morris
dbdbd906b7 Issue 547 - Decompress kmz files 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2453 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-08 00:29:25 +00:00
Tom Morris
4a99abf25d Isse 542 - allow integers to be converted to dates 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2450 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-03 21:36:36 +00:00
Tom Morris
5d080e5b3e Wrap if statement in a block to avoid future problems. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2447 7d457c2a-affb-35e4-300a-418c747d4874
 2012-03-01 18:10:59 +00:00
Tom Morris
c583ad4367 Issue 537 - Try to convert to Long first before converting to Double. Matches behavior on import. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2446 7d457c2a-affb-35e4-300a-418c747d4874
 2012-02-26 17:27:00 +00:00
Tom Morris
190e817fb8 Protect against NullPointerException 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2444 7d457c2a-affb-35e4-300a-418c747d4874
 2012-02-22 20:06:03 +00:00
David Huynh
e21ae32722 Make sure project ID is completely numeric. Slightly better error reporting on project page when project ID is not valid. 
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2441 7d457c2a-affb-35e4-300a-418c747d4874
 2012-01-29 21:16:13 +00:00