Commit Graph

41 Commits

Author SHA1 Message Date
Lisa Chandra
947356ddad
[FEAT]Adds new options for split (#2471)
* added options ui

* added definition for both separators

* added tests

* removed definitions from backend and added them to frontend

* added reverse order and handling for accented characters

* added tests for accented characters and reverse split

* fixed build errors

* unicode character ranges instead

* added examples
2020-06-15 19:30:18 +02:00
viniciusbds
496f1fd2d0 Fix bug when accessing empty list 2019-10-02 08:56:22 -03:00
Antonin Delpeuch
e4e73d0b92 Migrate importers to use Jackson classes 2018-11-30 19:23:08 +09:00
Antonin Delpeuch
5cf58d874b Fix deserialization of MassEditOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
d012ea7f8d Remove reconstruct method on operations 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
8f9004f7ee Jackson deserialization for TransposeRowsIntoColumnsOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
6358a4aeb5 Jackson deserialization for TransposeColumnsIntoRowsOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
8647e3d586 Jackson deserialization for TextTransformOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
6ad6b45048 Jackson deserialization for MultiValuedCellSplitOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
d493bf0720 Jackson deserialization for MassEditOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
50ee6ecc61 Jackson deserialization for KeyValueColumnizeOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
0cf8c2dd04 Jackson deserialization for MultiValuedCellJoinOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
379a12d349 Jackson deserialization for FillDownOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
3aa4e1a544 Jackson deserialization for BlankDownOperation 2018-11-30 19:22:21 +09:00
Antonin Delpeuch
1790799bfe Remove Jsonizable interface and write methods 2018-11-30 19:21:21 +09:00
Antonin Delpeuch
539585fc5e Jackson serialization for cell operations 2018-11-30 19:19:01 +09:00
Antonin Delpeuch
1eb9ef78ef Fix cell index computation in filldown/blankdown 2018-11-03 17:10:23 +00:00
Antonin Delpeuch
fbc9b27640 Store engine configuration as EngineConfig rather than JSONObject 2018-09-05 14:49:39 +01:00
Owen Stephens
0e954bdb68 Use stringutils to create 'from' string for non-blank mass edits 2018-06-23 00:18:00 +01:00
Owen Stephens
689442fff2 If only one value and that's zero length set fromBlank to true
Added whitespace
2018-06-08 09:40:48 +01:00
Antonin Delpeuch
53c12ca2df Fix FillDown and BlankDown operations on records.
Closes #472.
2018-04-21 22:11:54 +02:00
Jacky
c4b0ff6bea data package metadata (#1398)
* fix the appbundle issue #1209

* fix #1162

allow the JRE 9

* fix the package declarations

* remove the _ from the method name

* use the explicit scoping

* remote extra ;

* fix issued from codacy

* fix issued from codacy

* add preferences link to the index page

* handle the empty user metadata

* fix 'last modified' sorting issue #1307

* prevent overflow of the table. issue #1306

* add isoDateParser to sort the date

* prevent overflow of the project index

* remove sorter arrow for action columns

* disable editing the internal metadata

* adjust the width of the table

* change MetaData to Metadata

* change the filed name from rowNumber to rowCount

* put back the incidently deleted gitignore

* add double quote to prevent word splitting

* UI improvement on metadata view and project list view

* remove the date field in metadata

* message notification of the free RAM. Issue #1295

* UI tuning for metadata view

* shorten the ISO date to locale date format

* Added translation using Weblate (Portuguese (Brazil))

* remove the rename link

* Ignore empty language files introduced by Weblate

* Add UI for Invert text filter

* Backend support for Inverting Text search facets

* Fix reset on text search facet

* More succinct return statements

* add tests for SetProjectMetadataCommand

* Tidying up for Codacy

* Added Tests for TextSearchFilter

* Corrections for Codacy

* More code tidy up

* let the browser auto fit the table cell when resizing/zooming

* fix import multiple excel with mulitple sheets issue #1328

* check if the project has the userMetadata

* fix the unit test
support multi files with multi tables for open office

* prevent the same key for user metadata

* replace _ with variable for exception

* fix the no-undef issue

* to adjust the width of transform dialog. issue #1332

* fix the row count refresh issue

* extract method

* move the log message

* cosmatic changes for codacy

* fix typo

* bump to version 2.8

* .gitignore is now working

* preview stage won't have the metadata populated, so protect NPE

* Update README.md

No more direct link to the last version tag, which will avoid having to think of updating the readme

* refacotring the ProjectMetadata class

* introduce the IMetadata interface

* create submodule of dataschema

* add back

* setup lib for dataschema; upgrade the apache lang to lang3

* replace escape* functions from apache lang3

* replace the ProjectMetadata with IMetadata interface

* add missing jars

* set the IMetadata a field of Project

* remove PreferenceStore out of Project model

* fix test SetProjectMetadataCommandTests by casting

* introdcue the AbstractMetadata

* introdcue the AbstractMetadata

* reorganize the metadata package

* allow have mulitiple metadata for a project

* support for mulitple metadata format

* remove jdk7 since 'table schema' java implmentation only support jdk8+

* set execute permission for script

* fix the Unit Test after Metadata refactoring

* restore the apache lang2.5 since jetty 6.1.22 depend on it

* add commons lang 2.5 jar

* git submodule add  https://github.com/frictionlessdata/datapackage-java

* remove the metadata parameter from the ProjectManager.registerProject method

* remove hashmap _projectsMetadata field from the ProjectManager and FileProjectManager

* init the Project.metadataMap

* fix Unit Test

* restore the ProjectMetaData map to ProjectManager

* put the ProjectMetaDta in place for ProjectManager and Project object

* check null of singleton instead of create a constructor just for test

* load the data package metadata

* importing data package

* importing data package

* encapsulate the Package class into DataPackageMetadata

* user _ to indicate the class fields

* introduce base URL in order to download the data files

* import data package UI and draft backend

* import data package UI

* fix typo

* download the data set pointed from metadata resource

* save and load the data package metadata

* avoid magic string

* package cleanup

* set the java_version to 1.8

* set the min jdk to 1.8

* add the 3rd party src in the build.xml

* skip the file selection page if only 1 DATA file

* add files structure for json editor

* seperate out the metadata file from the retrival file list

* rename the OKF_METADATA to DATAPACKAGE_METADATA

* clean up

* implement GetMetadateCommand class

* display the metadata in json format

* git submodule update --remote --merge

* adjust the setting after pulling from datapackage origin

* fix the failed UT DateExtensionTests.testFetchCounts due to new json jar json-20160810.jar will complain: JSONObject["float"] not a string.

* clean up the weird loop array syntax get complained

* remove the unused constant

* export in data package format

* interface cleanup

* fix UT

* edit the metadata

* add UT for SetMetadataCommand

* fix UT for SetMetadataCommand

* display the data package metadata link on the project index page

* update submodule

* log the exceptions

* Ajv does not work properly, use the back end validation instead

* enable the validation for jsoneditor

* first draft of the data validation

* create a map to hold the constraint and its handler

* rename

* support for minLength and maxLength from spec

* add validate command

* test the opeation instead of validate command

* rename the UT

* format the error message and push to the report

* fix row number

* add resource bundle for validator

* inject the code of the constrains

* make the StrSubstitutor works

* extract the type and format information

* add the customizedFormat to interface to allow format properly

* get rid of magic string

* take care of missing parts of the data package

* implement RequiredConstraint

* patch for number type

* add max/min constraints

* get the constrains directly from field

* implement the PatternConstraint

* suppress warning

* fix the broken UT when expecting 2 digits fraction

* handle the cast and type properly

* fix the missing resource files for data package when run from command line

* use the copy instead of copydir

* add script for appveyor

* update script for appveyor

* do recursive clone

* correct the git url

* fix clone path

* clone folder option does not work

* will put another PR for this. delete for now

* revert the interface method name

* lazy loading the project data

* disable the validate menu for now

* add UT

* assert UTs

* add UT

* fix #1386

* remove import

* test the thread

* Revert "test the thread"

This reverts commit 779214160055afe3ccdcc18c57b0c7c72e87c824.

* fix the URLCachingTest UT

* define the template data package

* tidy up the metadata interface

* check the http response code

* fix the package

* display user friendly message when URL path is not reachable

* populate the data package schema

* Delete hs_err_pid15194.log

* populate data package info

* add username  preference and it will be pulled as the creator of the metadata

* undo the project.updateColumnChange() and start to introduce the fields into the existing core model

* tightly integrate the data package metadata

* tightly integrate the data package metadata for project level

* remove the submodule

* move the edit botton

* clean up build

* load the new property

* load the project metadata

* fix issues from codacy

* remove unused fields and annotation

* check the http response code firstly

* import zipped data package

* allow without keywords

* process the zip data package from url

* merge the tags

* check store firstly

* remove the table schema src

* move the json schema files to schema dir

* add comment

* add comment

* remove git moduels

* add incidently deleted file

* fix typo

* remove SetMetadataCommand

* revert change

* merge from master
2018-02-02 13:24:19 +00:00
Owen Stephens
224210625d Remove automatic trim of split values 2017-10-24 08:28:37 +01:00
Owen Stephens
46c3ec100e Remove unused local variables and imports 2017-10-23 08:36:08 +01:00
Antonin Delpeuch
23b643426a Fix Codacy warnings in MultiValuedCellSplitOperation 2017-10-23 08:41:14 +02:00
Owen Stephens
cccf1e55c9 Update split multi-valued cells to support split by regex and split by lengths 2017-10-22 23:54:18 +01:00
Qi Cui
773151380e fix #1138. column transpose 2016-08-24 13:56:35 -04:00
jackyq2015
7a2a0eb52f fixed issue #796 Columnize by key/value columns creates empty lines 2015-09-29 20:12:05 -04:00
Tom Morris
36dd95c263 Add TODO for record mode operation 2013-05-26 07:54:33 -04:00
Tom Morris
ca2e959957 FIXED - task 529: Add support for key/value transpose with only two columns as well as repeating key fields in a single record.
http://code.google.com/p/google-refine/issues/detail?id=529

git-svn-id: http://google-refine.googlecode.com/svn/trunk@2574 7d457c2a-affb-35e4-300a-418c747d4874
2012-10-05 23:31:25 +00:00
David Huynh
dbeaefb00b Minor bug fix to previous check-in: made sure blank cells in the 2 newly generated columns don't get filled in.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2368 7d457c2a-affb-35e4-300a-418c747d4874
2011-11-07 19:53:26 +00:00
David Huynh
d01745284b Added option to "transpose columns into rows" operation for filling in other columns.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2367 7d457c2a-affb-35e4-300a-418c747d4874
2011-11-07 19:39:23 +00:00
David Huynh
a35b9f53f7 Made operation "Transpose columns into rows" support the option of transposing into 2 new columns rather than just one.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2362 7d457c2a-affb-35e4-300a-418c747d4874
2011-11-06 02:50:33 +00:00
David Huynh
2f6b635f66 Added initial implementation of Key/value Columnize operation and command.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2356 7d457c2a-affb-35e4-300a-418c747d4874
2011-11-04 21:00:32 +00:00
David Huynh
f4b2ee3715 "Transpose columns into rows" operation now supports specifying the ending column to be the last column regardless of its name.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2337 7d457c2a-affb-35e4-300a-418c747d4874
2011-10-19 13:42:50 +00:00
Tom Morris
bde63ff417 Last set of indentation cleanups - no functional changes
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2211 7d457c2a-affb-35e4-300a-418c747d4874
2011-08-18 17:46:36 +00:00
Tom Morris
7fd6e22af4 Convert tabs to spaces. No functional changes.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2172 7d457c2a-affb-35e4-300a-418c747d4874
2011-08-02 20:26:32 +00:00
Tom Morris
123614539d Add missing @Override annotations
git-svn-id: http://google-refine.googlecode.com/svn/trunk@2171 7d457c2a-affb-35e4-300a-418c747d4874
2011-08-02 19:30:23 +00:00
David Huynh
1eebe2e4a3 Fixed transpose-rows-into-columns command, which previously duplicated columns that precede the column being transposed.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1734 7d457c2a-affb-35e4-300a-418c747d4874
2010-11-01 17:59:58 +00:00
David Huynh
5a17acfd70 Prepended license text to java source
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1613 7d457c2a-affb-35e4-300a-418c747d4874
2010-10-20 20:45:52 +00:00
David Huynh
edb23eb263 Changed Java packages com.google.gridworks.* to com.google.refine.* and modified other code just enough to start grefine up without error. Much remains to be done. Do not check out the code just yet.
git-svn-id: http://google-refine.googlecode.com/svn/trunk@1288 7d457c2a-affb-35e4-300a-418c747d4874
2010-09-22 17:04:10 +00:00