Commit Graph

135 Commits

Author SHA1 Message Date
Tom Morris
f88c0e3657
Preserve international characters in project/file names on import/export (#2720)
* Preserve international characters on import/export

Fixes #1352. Preserve non-ASCII characters in project names on
project creation and filenames on export.

Uses existing filename cleaner with the addition of a few
more characters from StackOverflow, plus "#" which messes
up the download URL. Also URIencode download URL.

Removes unused I18N-incompatible cleaning function from
Wikidata extension rather than fixing it.

* Use common name cleaner function

Also preview cleaned table name instead of raw name, so user can see it.
Also add a TODO for better preview of column names
2020-06-18 22:06:46 +02:00
james-cui
04055153a1
add archive column (#2573)
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
2020-06-15 19:56:00 +02:00
Tom Morris
03c860d961
Update jquery.i18n to 1.07 and fix non-English plural support (#2717)
* Update jquery.i18n to 1.07 and add missing rule parser

Fixes #2700
Adds missing CLDRPluralRuleParser.js so that plurals are supported.
Updates all files to jquery.i18n 1.07
Includes a bunch of specialty language support,
but only Finnish and Russian are loaded as examples.

* Add some missing translations, including plurals

Fix some cases of Javascript string concatenation and
plural conditionalization to demonstrate that plurals work
in both English and French now.

NOTE: Corresponding updates need to be made to all the other
language files since some keys were renamed or eliminated.
2020-06-14 10:23:23 +02:00
Nishtha
cccaf2992a
Trim input url whitespace (#2534)
Fixes #2516
2020-06-10 08:28:30 -04:00
chuhao zeng
1f0111eaed
Fix silent error in JSON/XML importers (#2414)
* Add error handler for parse error

* Add test for parsing json with incorrect strecture

* Enable localization from front-end

* Add methods to get localized error messages

* Update returned exception message

* Remove unused log and fix file diff issue

* Test auto build

* Refactor getOptions in newly created test

* Use new exception to unwrap original message

* Undo unexpected fix

* Remove unused lines

* Fix exception logic

* Fix typo
2020-03-27 09:41:49 +01:00
Lisa Chandra
ef8ad85c3c
Adds trim whitespace option to separator based files (#2408)
* added trim ui to csv importer

* added trim functionality

* trimStrings handler only for strings

* added test for trimStrings option in csv/tsv files

* made trim option enabled by default
2020-03-21 10:38:43 +00:00
Ekta Mishra
bf8af23493
Creates project from URL when press ENTER (#2444)
* Creates project from URL when press ENTER

solves #2437

* Added submit action
2020-03-20 22:35:40 +01:00
Lisa Chandra
fd2460914c
Fixes no selected record element error (#2407)
* fixes no record element error

* made relevant changes

* alert message from i18n instead

* removed the boolean variable
2020-03-14 14:36:54 +00:00
Lisa
b5282fcce7
corrected label for trim in xml parser ui (#2394) 2020-03-12 10:54:53 +00:00
Kush Trivedi
3fd3faca11
enh: URL checking for import web addresses (#2377)
* enh: URL checking for import web addresses

* Add the source for Regex

* Removing the console.log for production
2020-03-11 22:22:49 +00:00
Lisa
649ce93b3a
corrects wrong trim label for json importer config (#2385) 2020-03-11 09:41:16 +00:00
Thad Guidry
9a1d6a341a fix typo 2020-02-26 07:55:35 -06:00
Thad Guidry
4baba6b125
fix typo 2020-02-26 07:44:44 -06:00
Antonin Delpeuch
11c7788239 Fix CSRF token addition on import project form 2019-11-04 20:04:16 +00:00
Antonin Delpeuch
9ae6a7a581 Tie up CSRF tokens in the frontend 2019-10-15 12:07:14 +01:00
Antonin Delpeuch
5dc005749a Add CSRF protection to remaining commands 2019-10-15 12:07:13 +01:00
Antonin Delpeuch
3559eeb11f CSRF protection for project and recon commands 2019-10-15 12:07:12 +01:00
Antonin Delpeuch
a340c137d0 CSRF protection for OpenWorkspaceDirCommand and language loading 2019-10-15 12:07:04 +01:00
Antonin Delpeuch
f7177e670d Adapt frontend for CSRF in import dialog 2019-10-15 12:04:17 +01:00
Antonin Delpeuch
91cead27f8 CSRF protection for ImportingController 2019-10-14 16:24:26 +01:00
Antonin Delpeuch
70e37b9085 Add CSRF protection to cell, history, column and expr commands 2019-10-14 16:24:26 +01:00
Antonin Delpeuch
c078c3a81d Fix change of language. Closes #2158. 2019-09-12 11:53:21 +01:00
Antonin Delpeuch
4f43515ec9 Fix project metadata UI 2019-06-06 17:15:25 +01:00
Daniel Sader
d21ca26b2f Added Select All/Select None buttons to Excel importer 2019-04-01 14:53:50 -03:00
Antonin Delpeuch
a4117b470c Remove leftover merge conflicts 2018-11-30 19:39:42 +09:00
Antonin Delpeuch
5a4a79028f Revert "data package metadata (#1398)"
This reverts commit c4b0ff6bea.
2018-11-30 19:35:31 +09:00
Antonin Delpeuch
7dd4db1ca1 Fix i18n calls after migration 2018-11-28 23:22:38 +09:00
Antonin Delpeuch
10931e1e4d Migrate js to new localization syntax 2018-11-21 17:30:00 +00:00
Antonin Delpeuch
b46c5a7b4e Flatten remaining messages 2018-11-21 17:26:53 +00:00
Antonin Delpeuch
9bf0bb2154 Merge JS i18n strings with slash 2018-11-21 15:24:57 +00:00
xseris
f04b5ef048 moved input box below label 2018-09-13 17:10:04 +02:00
xseris
280650867e move "comma separated" near "Column values" 2018-09-13 16:25:04 +02:00
xseris
f288bc653e Added test, changed visualization 2018-09-13 14:45:38 +02:00
xseris
81e8e4c2f6 Fix visualization. Auto refresh preview on input change. 2018-09-13 11:25:07 +02:00
xseris
deab8ceeb0 custom column names for separator based importer 2018-09-12 15:45:52 +02:00
isaomatsunami
91c6c6c791 fix layout 2018-02-09 03:53:11 +09:00
Owen Stephens
0d04a25cf1
Merge pull request #1450 from OpenRefine/issue1448
Make Wikitext importer more robust
2018-02-07 17:41:23 +00:00
Thad Guidry
c83dbe3b23
Merge branch 'master' into stundzig-develop/1086-quotecharacter 2018-02-03 07:20:45 -06:00
Antonin Delpeuch
86be1c12e8 Expose quote character setting in UI 2018-02-03 09:01:07 +00:00
Jacky
c4b0ff6bea data package metadata (#1398)
* fix the appbundle issue #1209

* fix #1162

allow the JRE 9

* fix the package declarations

* remove the _ from the method name

* use the explicit scoping

* remote extra ;

* fix issued from codacy

* fix issued from codacy

* add preferences link to the index page

* handle the empty user metadata

* fix 'last modified' sorting issue #1307

* prevent overflow of the table. issue #1306

* add isoDateParser to sort the date

* prevent overflow of the project index

* remove sorter arrow for action columns

* disable editing the internal metadata

* adjust the width of the table

* change MetaData to Metadata

* change the filed name from rowNumber to rowCount

* put back the incidently deleted gitignore

* add double quote to prevent word splitting

* UI improvement on metadata view and project list view

* remove the date field in metadata

* message notification of the free RAM. Issue #1295

* UI tuning for metadata view

* shorten the ISO date to locale date format

* Added translation using Weblate (Portuguese (Brazil))

* remove the rename link

* Ignore empty language files introduced by Weblate

* Add UI for Invert text filter

* Backend support for Inverting Text search facets

* Fix reset on text search facet

* More succinct return statements

* add tests for SetProjectMetadataCommand

* Tidying up for Codacy

* Added Tests for TextSearchFilter

* Corrections for Codacy

* More code tidy up

* let the browser auto fit the table cell when resizing/zooming

* fix import multiple excel with mulitple sheets issue #1328

* check if the project has the userMetadata

* fix the unit test
support multi files with multi tables for open office

* prevent the same key for user metadata

* replace _ with variable for exception

* fix the no-undef issue

* to adjust the width of transform dialog. issue #1332

* fix the row count refresh issue

* extract method

* move the log message

* cosmatic changes for codacy

* fix typo

* bump to version 2.8

* .gitignore is now working

* preview stage won't have the metadata populated, so protect NPE

* Update README.md

No more direct link to the last version tag, which will avoid having to think of updating the readme

* refacotring the ProjectMetadata class

* introduce the IMetadata interface

* create submodule of dataschema

* add back

* setup lib for dataschema; upgrade the apache lang to lang3

* replace escape* functions from apache lang3

* replace the ProjectMetadata with IMetadata interface

* add missing jars

* set the IMetadata a field of Project

* remove PreferenceStore out of Project model

* fix test SetProjectMetadataCommandTests by casting

* introdcue the AbstractMetadata

* introdcue the AbstractMetadata

* reorganize the metadata package

* allow have mulitiple metadata for a project

* support for mulitple metadata format

* remove jdk7 since 'table schema' java implmentation only support jdk8+

* set execute permission for script

* fix the Unit Test after Metadata refactoring

* restore the apache lang2.5 since jetty 6.1.22 depend on it

* add commons lang 2.5 jar

* git submodule add  https://github.com/frictionlessdata/datapackage-java

* remove the metadata parameter from the ProjectManager.registerProject method

* remove hashmap _projectsMetadata field from the ProjectManager and FileProjectManager

* init the Project.metadataMap

* fix Unit Test

* restore the ProjectMetaData map to ProjectManager

* put the ProjectMetaDta in place for ProjectManager and Project object

* check null of singleton instead of create a constructor just for test

* load the data package metadata

* importing data package

* importing data package

* encapsulate the Package class into DataPackageMetadata

* user _ to indicate the class fields

* introduce base URL in order to download the data files

* import data package UI and draft backend

* import data package UI

* fix typo

* download the data set pointed from metadata resource

* save and load the data package metadata

* avoid magic string

* package cleanup

* set the java_version to 1.8

* set the min jdk to 1.8

* add the 3rd party src in the build.xml

* skip the file selection page if only 1 DATA file

* add files structure for json editor

* seperate out the metadata file from the retrival file list

* rename the OKF_METADATA to DATAPACKAGE_METADATA

* clean up

* implement GetMetadateCommand class

* display the metadata in json format

* git submodule update --remote --merge

* adjust the setting after pulling from datapackage origin

* fix the failed UT DateExtensionTests.testFetchCounts due to new json jar json-20160810.jar will complain: JSONObject["float"] not a string.

* clean up the weird loop array syntax get complained

* remove the unused constant

* export in data package format

* interface cleanup

* fix UT

* edit the metadata

* add UT for SetMetadataCommand

* fix UT for SetMetadataCommand

* display the data package metadata link on the project index page

* update submodule

* log the exceptions

* Ajv does not work properly, use the back end validation instead

* enable the validation for jsoneditor

* first draft of the data validation

* create a map to hold the constraint and its handler

* rename

* support for minLength and maxLength from spec

* add validate command

* test the opeation instead of validate command

* rename the UT

* format the error message and push to the report

* fix row number

* add resource bundle for validator

* inject the code of the constrains

* make the StrSubstitutor works

* extract the type and format information

* add the customizedFormat to interface to allow format properly

* get rid of magic string

* take care of missing parts of the data package

* implement RequiredConstraint

* patch for number type

* add max/min constraints

* get the constrains directly from field

* implement the PatternConstraint

* suppress warning

* fix the broken UT when expecting 2 digits fraction

* handle the cast and type properly

* fix the missing resource files for data package when run from command line

* use the copy instead of copydir

* add script for appveyor

* update script for appveyor

* do recursive clone

* correct the git url

* fix clone path

* clone folder option does not work

* will put another PR for this. delete for now

* revert the interface method name

* lazy loading the project data

* disable the validate menu for now

* add UT

* assert UTs

* add UT

* fix #1386

* remove import

* test the thread

* Revert "test the thread"

This reverts commit 779214160055afe3ccdcc18c57b0c7c72e87c824.

* fix the URLCachingTest UT

* define the template data package

* tidy up the metadata interface

* check the http response code

* fix the package

* display user friendly message when URL path is not reachable

* populate the data package schema

* Delete hs_err_pid15194.log

* populate data package info

* add username  preference and it will be pulled as the creator of the metadata

* undo the project.updateColumnChange() and start to introduce the fields into the existing core model

* tightly integrate the data package metadata

* tightly integrate the data package metadata for project level

* remove the submodule

* move the edit botton

* clean up build

* load the new property

* load the project metadata

* fix issues from codacy

* remove unused fields and annotation

* check the http response code firstly

* import zipped data package

* allow without keywords

* process the zip data package from url

* merge the tags

* check store firstly

* remove the table schema src

* move the json schema files to schema dir

* add comment

* add comment

* remove git moduels

* add incidently deleted file

* fix typo

* remove SetMetadataCommand

* revert change

* merge from master
2018-02-02 13:24:19 +00:00
Antonin Delpeuch
843641ca74 Parse headers just like other cells in Wikitable importer
This makes the column names a bit less meaningful,
but this is necessary to handle cases where headers
are inserted anywhere in the table.
2018-02-02 10:48:31 +00:00
xseris
ee8ed8f563 Refresh tag cell 2017-12-08 00:46:59 +01:00
xseris
b16941777e Modified tags in metadata dialog and moved column right to name 2017-12-07 23:45:36 +01:00
xseris
f21230804b Fixed Mockito tests 2017-12-01 17:00:45 +01:00
xseris
d2648d8db8 Fix tests 2017-11-28 00:16:08 +01:00
xseris
cecbfb486b Remove delete all projects option 2017-11-24 20:29:09 +01:00
xseris
a3132eeeb3 implemented tag system 2017-11-24 20:19:49 +01:00
xseris
4f67c632d0 delete all projects 2017-11-23 17:36:46 +01:00
Thad Guidry
daca1836e9
Merge pull request #1342 from jackyq2015/feature/metadata
fix the row count refresh issue.  We'll work on adding a Delete Metadata option in the next milestone.
2017-11-18 08:30:32 -06:00
Jacky
b83961d2c6 cosmatic changes for codacy 2017-11-17 17:14:19 -05:00