/[metamod]/trunk/upload
ViewVC logotype

Log of /trunk/upload

View Directory Listing Directory Listing


Sticky Revision:

Revision 2867 - Directory Listing
Modified Tue Nov 3 14:30:22 2015 UTC (4 years, 9 months ago) by geira
removed a million deprecated "use encoding" statements, hoping tests will pass

Revision 2824 - Directory Listing
Modified Mon Jun 29 13:18:21 2015 UTC (5 years, 1 month ago) by egils
Allowing netCDF4 files to be uploaded

Revision 2690 - Directory Listing
Modified Wed Jun 11 14:44:55 2014 UTC (6 years, 2 months ago) by geira
fixed ftp_process_hour which didn't handle errors after last rewrite
added ctrl-c signal handler to ftp_monitor so doesn't quit in middle of job
added basic ftp processing unit tests (currently only sees if doesn't die)

Revision 2668 - Directory Listing
Modified Fri May 23 09:00:31 2014 UTC (6 years, 2 months ago) by geira
now runs from outside of dir, takes multiple file parameters, more docs

Revision 2661 - Directory Listing
Modified Tue May 20 14:28:22 2014 UTC (6 years, 2 months ago) by geira
improved docs, checks, error msgs and user interface

Revision 2574 - Directory Listing
Modified Wed Nov 20 17:08:02 2013 UTC (6 years, 8 months ago) by geira
username arg check fixed

Revision 2573 - Directory Listing
Modified Wed Nov 20 17:01:43 2013 UTC (6 years, 8 months ago) by geira
added option to read OPERATOR_EMAIL automatically

Revision 2572 - Directory Listing
Modified Wed Nov 20 16:51:49 2013 UTC (6 years, 8 months ago) by geira
even more username logging

Revision 2571 - Directory Listing
Modified Wed Nov 20 16:47:57 2013 UTC (6 years, 8 months ago) by geira
more tuning of operator user account

Revision 2568 - Directory Listing
Modified Wed Nov 20 16:13:37 2013 UTC (6 years, 8 months ago) by geira
bug: missing perl shebang

Revision 2559 - Directory Listing
Modified Wed Nov 20 14:31:35 2013 UTC (6 years, 8 months ago) by geira
fixed lib path so now works when called from outside script dir

Revision 2520 - Directory Listing
Modified Mon Oct 14 13:50:20 2013 UTC (6 years, 10 months ago) by geira
fixed double transform button
more work in progress on ncdigest.pl
proper version number, encoding fix in MetamodWeb.pm
docs updated

Revision 2519 - Directory Listing
Modified Fri Oct 11 11:31:55 2013 UTC (6 years, 10 months ago) by geira
digest_nc superceded by ncdigest with updated parameter syntax
podified docs

Revision 2470 - Directory Listing
Modified Fri Sep 20 10:38:30 2013 UTC (6 years, 10 months ago) by geira
more docs, fix for missing config init (doesn't seem to have run for some time...)

Revision 2469 - Directory Listing
Modified Fri Sep 20 10:13:27 2013 UTC (6 years, 10 months ago) by geira
updated & podified docs

Revision 2345 - Directory Listing
Modified Fri Mar 8 12:27:56 2013 UTC (7 years, 5 months ago) by geira
merged r2344 into trunk

Revision 2232 - Directory Listing
Modified Fri Mar 30 13:30:41 2012 UTC (8 years, 4 months ago) by geira
typo

Revision 2230 - Directory Listing
Modified Fri Mar 30 13:17:06 2012 UTC (8 years, 4 months ago) by geira
optimistic hack to try stopping upload/scripts/add_file_to_queue.pl from bombing when Metamod::Queue has no config during test_application

Revision 2225 - Directory Listing
Modified Fri Mar 30 10:26:51 2012 UTC (8 years, 4 months ago) by geira
setting proper exec priv on scripts which now fail under test_application

Revision 2220 - Directory Listing
Modified Thu Mar 29 15:34:39 2012 UTC (8 years, 4 months ago) by geira
removed some more dependencies on target

Revision 2129 - Directory Listing
Modified Mon Jan 9 16:13:39 2012 UTC (8 years, 7 months ago) by geira
fixed buggy ftp daemon invocation; general cleanup

Revision 2082 - Directory Listing
Modified Thu Nov 3 17:10:29 2011 UTC (8 years, 9 months ago) by geira
more logging and docs

Revision 2079 - Directory Listing
Modified Wed Nov 2 12:32:01 2011 UTC (8 years, 9 months ago) by geira
reverted experimental feature for sharing userbase queues
from now on every instance must have its own userbase

Revision 2042 - Directory Listing
Modified Thu Oct 6 11:15:34 2011 UTC (8 years, 10 months ago) by geira
added pod2usage, fixed check for config path

Revision 2037 - Directory Listing
Modified Thu Oct 6 10:32:54 2011 UTC (8 years, 10 months ago) by geira
now implemented both test (run once) and normal (repeat) foreground operation

Revision 2025 - Directory Listing
Modified Wed Oct 5 15:33:03 2011 UTC (8 years, 10 months ago) by geira
cleanup of init code, removal of old junk
test run currently not implemented

Revision 2009 - Directory Listing
Modified Wed Oct 5 12:01:38 2011 UTC (8 years, 10 months ago) by geira
revised parameter list to conform with new standard

Revision 2000 - Directory Listing
Modified Tue Oct 4 14:12:49 2011 UTC (8 years, 10 months ago) by geira
queue system now more abstracted, using coalesce values to match application_id
this prevents race conditions when multiple sites sharing same user database

Revision 1944 - Directory Listing
Modified Fri Sep 9 08:13:54 2011 UTC (8 years, 11 months ago) by oysteint
Moved files to config/etc/

Revision 1943 - Directory Listing
Modified Fri Sep 9 08:13:32 2011 UTC (8 years, 11 months ago) by oysteint
Updated NcDigest to use path_to_config_file() to get the configuration files.

Removed $etcdirectory as a parameter to digest().

Revision 1935 - Directory Listing
Modified Thu Sep 8 10:36:49 2011 UTC (8 years, 11 months ago) by oysteint
Merged in changes from branches/remove_target

Revision 1894 - Directory Listing
Modified Wed Aug 17 11:41:52 2011 UTC (8 years, 11 months ago) by egils
Removed extra timestamp on log messages to avoid reporting from continous integration script

Revision 1893 - Directory Listing
Modified Wed Aug 17 11:11:18 2011 UTC (8 years, 11 months ago) by oysteint
Removed PHP and CGI scripts that are no longer in use.

Revision 1876 - Directory Listing
Modified Thu Aug 11 12:26:28 2011 UTC (9 years ago) by egils
Small bugfix

Revision 1872 - Directory Listing
Modified Wed Aug 10 08:32:47 2011 UTC (9 years ago) by egils
Reshuffled lib pathes in use lib to avoid preference of ../../common/lib

Revision 1869 - Directory Listing
Modified Tue Aug 9 11:38:53 2011 UTC (9 years ago) by egils
Added OPERATOR_INSTITUTION to all master_config.txt files. This will be used as intitution name
for the Admin user when the user database is initialized. It will also be used as a directory 
name in the web upload area.

Major change to the test_application.sh script, using the userbase_add_datasets.pl script to add
datasets to the user database, and the add_file_to_queue.pl script to add netCDF files to the
web upload queue.

Revision 1868 - Directory Listing
Modified Mon Aug 8 13:32:41 2011 UTC (9 years ago) by geira
- fixed upload so now sends emails again
- fixed error handling in upload_monitor
- removed obsolete Metamod::UploadMonitor
- added some docs, removed debug output


Revision 1862 - Directory Listing
Modified Fri Jul 22 12:42:16 2011 UTC (9 years ago) by egils
New script for adding nc-file to upload queue. Used for testing.

Revision 1861 - Directory Listing
Modified Thu Jul 21 15:47:39 2011 UTC (9 years ago) by geira
ftp_monitor rewritten to use new UploadHelper OO API, with crontab style scheduling in master_config
upload job type keywords now consistent with traditional usage (TAF, WEB, FTP)
also some documentation and error checking added


Revision 1856 - Directory Listing
Modified Tue Jul 12 08:26:00 2011 UTC (9 years, 1 month ago) by oysteint
Added conf_digest_nc.xml and usererrors.conf to the source directory to allow automatic test for
UploadHelper to run from source.

Revision 1854 - Directory Listing
Modified Fri Jul 8 13:14:01 2011 UTC (9 years, 1 month ago) by oysteint
Removed import of UploadMonitor since it is no longer needed.

Made the path to common/lib absolute.

Revision 1848 - Directory Listing
Modified Fri Jul 8 11:26:16 2011 UTC (9 years, 1 month ago) by oysteint
Moved the code from print_usererrors.pl into a module for easier calling form other modules.

Revision 1828 - Directory Listing
Modified Fri Jul 1 08:02:33 2011 UTC (9 years, 1 month ago) by geira
removed stupid debug output

Revision 1823 - Directory Listing
Modified Thu Jun 30 16:01:34 2011 UTC (9 years, 1 month ago) by geira
MAJOR CHANGES TO IMPORTS
* harvester now using crontab-like scheduling (via POE)
* upload_monitor rewritten to using event queue, now handles only web uploads
* ftp processing moved to new daemon ftp_monitor (soon to implement POE scheduling)


Revision 1822 - Directory Listing
Modified Thu Jun 30 15:58:11 2011 UTC (9 years, 1 month ago) by geira
MAJOR CHANGES TO IMPORTS
* harvester now using crontab-like scheduling (via POE)
* upload_monitor rewritten to using event queue, now handles only web uploads
* ftp processing moved to new daemon ftp_monitor (soon to implement POE scheduling)


Revision 1812 - Directory Listing
Modified Tue Jun 28 15:50:59 2011 UTC (9 years, 1 month ago) by geira
main_loop moved back to daemon script

Revision 1788 - Directory Listing
Modified Thu Jun 23 07:57:09 2011 UTC (9 years, 1 month ago) by egils
Improved project handling in PMH, bugfix in NcDigest.pm (merge from branch 2.8)

Revision 1784 - Directory Listing
Modified Tue Jun 21 12:42:20 2011 UTC (9 years, 1 month ago) by geira
moduleififed upload_monitor; moved Uploadutils.pm to lib dir

Revision 1783 - Directory Listing
Modified Tue Jun 21 12:40:22 2011 UTC (9 years, 1 month ago) by geira
moduleififed upload_monitor; moved Uploadutils.pm to lib dir

Revision 1741 - Directory Listing
Modified Thu May 26 10:59:48 2011 UTC (9 years, 2 months ago) by geira
removed warning since warnings = fatal errors in severity...

Revision 1738 - Directory Listing
Modified Tue May 24 10:35:13 2011 UTC (9 years, 2 months ago) by geira
generate explicit error msg since i don't trust $userbase->get_exception()

Revision 1735 - Directory Listing
Modified Fri May 20 06:54:58 2011 UTC (9 years, 2 months ago) by geira
removed next from do block


Revision 1731 - Directory Listing
Modified Thu May 19 09:21:15 2011 UTC (9 years, 2 months ago) by geira
no longer complain over admin users w/o institution 


Revision 1706 - Directory Listing
Modified Mon May 2 14:46:09 2011 UTC (9 years, 3 months ago) by heikok
fixing $xmlpath definition even if $destination_dir needs to be created

Revision 1702 - Directory Listing
Modified Mon May 2 09:37:17 2011 UTC (9 years, 3 months ago) by heikok
write log-warning for failing shell commands

Revision 1660 - Directory Listing
Modified Fri Apr 15 11:01:52 2011 UTC (9 years, 4 months ago) by egils
Bugfix

Revision 1659 - Directory Listing
Modified Fri Apr 15 10:49:53 2011 UTC (9 years, 4 months ago) by egils
More info reporting to metamod log

Revision 1657 - Directory Listing
Modified Fri Apr 15 06:41:05 2011 UTC (9 years, 4 months ago) by egils
Added code for creating directory path to dataset in OPENDAP_DIRECTORY

Revision 1633 - Directory Listing
Modified Wed Apr 6 11:22:01 2011 UTC (9 years, 4 months ago) by geira
all perl scripts set executable so can run from src

Revision 1569 - Directory Listing
Modified Fri Mar 25 09:52:58 2011 UTC (9 years, 4 months ago) by geira
removed PHP files from filelist so will no longer be copied to target

also changed Apache config so now everything will be proxied to catalyst
except when spec'd otherwise (static, pmh, upload errors)

Revision 1547 - Directory Listing
Modified Thu Mar 17 10:08:38 2011 UTC (9 years, 4 months ago) by oysteint
Added writing the NC destination file location to the digest_nc file for level 2 datasets.

Revision 1546 - Directory Listing
Modified Thu Mar 17 10:06:51 2011 UTC (9 years, 4 months ago) by oysteint
Removed WMS_URL from the list of dataset parameters to retrieve.

Revision 1388 - Directory Listing
Modified Wed Feb 16 16:04:29 2011 UTC (9 years, 5 months ago) by geira
hopefully fixed situation where printing debug to stderr in scripts is no longer treated as a fatal error
also some podification and perltidying

Revision 1368 - Directory Listing
Modified Tue Feb 15 08:05:06 2011 UTC (9 years, 5 months ago) by oysteint
Added 'data_file_location' and 'data_file_size' to the metadata.

Revision 1362 - Directory Listing
Modified Mon Feb 14 15:10:21 2011 UTC (9 years, 6 months ago) by geira
sort-of-working versions of file-upload and newfiles rewritten for catalyst

Revision 1307 - Directory Listing
Modified Wed Jan 26 13:59:56 2011 UTC (9 years, 6 months ago) by geira
when sending email, read server name from config, not from url
(might be a proxy server, certainly not localhost)


Revision 1306 - Directory Listing
Modified Wed Jan 26 10:55:30 2011 UTC (9 years, 6 months ago) by egils
Dataset key no longer required field in Dataset Administration form

Revision 1302 - Directory Listing
Modified Tue Jan 25 15:53:32 2011 UTC (9 years, 6 months ago) by geira
podified and perltidied


Revision 1301 - Directory Listing
Modified Tue Jan 25 10:08:48 2011 UTC (9 years, 6 months ago) by geira
podified & reindented

Revision 1298 - Directory Listing
Modified Mon Jan 24 13:17:21 2011 UTC (9 years, 6 months ago) by egils
Ported r1293 from 2.6 to trunk

Revision 1297 - Directory Listing
Modified Mon Jan 24 13:16:00 2011 UTC (9 years, 6 months ago) by egils
Ported r1247 from 2.6 to trunk

Revision 1296 - Directory Listing
Modified Mon Jan 24 13:14:46 2011 UTC (9 years, 6 months ago) by egils
Ported r1227 from 2.6 to trunk

Revision 1295 - Directory Listing
Modified Mon Jan 24 13:13:14 2011 UTC (9 years, 6 months ago) by egils
Ported r1213 from 2.6 to trunk

Revision 1294 - Directory Listing
Modified Mon Jan 24 13:05:07 2011 UTC (9 years, 6 months ago) by egils
Ported r1199 from 2.6 to trunk

Revision 1290 - Directory Listing
Modified Tue Jan 18 10:48:15 2011 UTC (9 years, 6 months ago) by egils
Corrected key value for getting user name from user database ('name' to 'u_name')

Revision 1089 - Directory Listing
Modified Wed Nov 24 13:00:43 2010 UTC (9 years, 8 months ago) by egils
Improved README files. They are now more or less correct for the 2.6 release. Removed LOGFILE from master_config files

Revision 1066 - Directory Listing
Modified Fri Nov 19 13:57:23 2010 UTC (9 years, 8 months ago) by egils
Fixed wrong array key for retrieving URL to error description

Revision 1061 - Directory Listing
Modified Wed Nov 17 14:09:58 2010 UTC (9 years, 8 months ago) by egils
Moved load_userbase.pl from BASE to UPLOAD. README improvements

Revision 1022 - Directory Listing
Modified Wed Nov 10 15:14:43 2010 UTC (9 years, 9 months ago) by heikok
ensuring that iso19139 datasets validate even if date
resolution is set to seconds
making sure that SRU and OAI give same identifier in iso-document
fixing docs in ncExtractLatLong

Revision 1000 - Directory Listing
Modified Tue Oct 26 12:15:00 2010 UTC (9 years, 9 months ago) by egils
Fixed bug: erroneous propagation of THREDDS catalog changes to dataref elements in xml files

Revision 975 - Directory Listing
Modified Mon Oct 18 12:33:22 2010 UTC (9 years, 9 months ago) by heikok
remove no longer used datestamp file

Revision 962 - Directory Listing
Modified Fri Oct 15 12:58:02 2010 UTC (9 years, 10 months ago) by heikok
enabling THREDDS_DATAREF substitution for WMS links

Revision 946 - Directory Listing
Modified Thu Oct 14 08:18:52 2010 UTC (9 years, 10 months ago) by egils
Bugfixes: Some updates in user database were not committed, password changes when logging into Upload were not registered in User database.

Revision 924 - Directory Listing
Modified Wed Oct 6 06:37:35 2010 UTC (9 years, 10 months ago) by egils
Bugfix, added comment

Revision 902 - Directory Listing
Modified Fri Oct 1 12:36:23 2010 UTC (9 years, 10 months ago) by egils
Changes in the Upload module in preparation for the 2.6 release. User information taken from the
User database. No longer from the webrun/u1 directory. Session handling based on file names in
webrun/u1 and webrun/u2 are kept as is.

Revision 865 - Directory Listing
Modified Mon Sep 20 13:37:32 2010 UTC (9 years, 10 months ago) by heikok
don't notify web-system with files moved to problem-dir

Revision 857 - Directory Listing
Modified Fri Sep 17 12:17:18 2010 UTC (9 years, 10 months ago) by heikok
moving progress_report to logger->debug

Revision 856 - Directory Listing
Modified Fri Sep 17 12:00:05 2010 UTC (9 years, 10 months ago) by heikok
moving progress_report to logger->debug

Revision 855 - Directory Listing
Modified Fri Sep 17 11:10:14 2010 UTC (9 years, 10 months ago) by heikok
better handling of line-feeds in error messages
removing upload-monitor.out

Revision 853 - Directory Listing
Modified Fri Sep 17 10:47:14 2010 UTC (9 years, 10 months ago) by heikok
log4perl in Uploadutils
unique constraint in sru-contact_meta

Revision 786 - Directory Listing
Modified Fri Sep 3 10:44:35 2010 UTC (9 years, 11 months ago) by heikok
fixing missing variables: PMH* OPENDAP_BASEDIR

Revision 723 - Directory Listing
Modified Fri Jul 16 08:38:52 2010 UTC (10 years, 1 month ago) by oysteint
Refactored digest_nc.pl by moving most of the code to MetNo::NcDigest. Moving code from the script to a module should simplify creating unit tests.

Revision 695 - Directory Listing
Modified Wed Jul 7 12:52:53 2010 UTC (10 years, 1 month ago) by heikok
connecting scripts to Log4perl

Revision 640 - Directory Listing
Modified Tue May 11 10:43:03 2010 UTC (10 years, 3 months ago) by egils
New standard name table (v13, 11 March 2010)

Revision 631 - Directory Listing
Modified Fri Apr 23 06:25:55 2010 UTC (10 years, 3 months ago) by egils
Better control of links to THREDDS using THREDDS_DATASET_PREFIX

Revision 630 - Directory Listing
Modified Thu Apr 22 15:18:26 2010 UTC (10 years, 3 months ago) by heikok
Clearing datasets instead of deleting. Delete will remove manually entered information in .xmd files.

Revision 626 - Directory Listing
Modified Tue Apr 20 10:56:22 2010 UTC (10 years, 3 months ago) by egils
Introduced OPENDAP_BASEDIR

Revision 621 - Directory Listing
Modified Fri Apr 9 07:33:10 2010 UTC (10 years, 4 months ago) by egils
Small bugfixes

Revision 617 - Directory Listing
Modified Thu Apr 8 06:54:16 2010 UTC (10 years, 4 months ago) by egils
Ported revisions 586,598 and 600 from branches/2.4

Revision 610 - Directory Listing
Modified Tue Apr 6 12:59:31 2010 UTC (10 years, 4 months ago) by heikok
adding real-world example and values for ncWmsSetup

Revision 608 - Directory Listing
Modified Tue Apr 6 12:08:58 2010 UTC (10 years, 4 months ago) by heikok
smaller fixes to get GetWMSSetup to work

Revision 579 - Directory Listing
Modified Wed Mar 10 13:51:08 2010 UTC (10 years, 5 months ago) by egils
Minor bugfix

Revision 573 - Directory Listing
Modified Thu Mar 4 08:53:10 2010 UTC (10 years, 5 months ago) by egils
Allowed registering gzipped .nc-files

Revision 569 - Directory Listing
Modified Fri Feb 26 12:33:29 2010 UTC (10 years, 5 months ago) by egils
Minor text improvement

Revision 568 - Directory Listing
Modified Fri Feb 26 11:35:32 2010 UTC (10 years, 5 months ago) by egils
Added support for registering files in subdirectories at any depth

Revision 556 - Directory Listing
Modified Tue Feb 16 11:23:51 2010 UTC (10 years, 5 months ago) by egils
Added support for providing a WMS url for each dataset

Revision 553 - Directory Listing
Modified Thu Feb 11 14:54:28 2010 UTC (10 years, 6 months ago) by egils
Follow-up on retrieving info from webrun/u1 user files

Revision 552 - Directory Listing
Modified Thu Feb 11 13:57:17 2010 UTC (10 years, 6 months ago) by heikok
MERGE Metamod-postgis-dev2.5 into 2.x trunk before releasing 2.4

Revision 551 - Directory Listing
Modified Thu Feb 11 13:30:16 2010 UTC (10 years, 6 months ago) by egils
Improved maintenence of user files on webrun/u1

Revision 541 - Directory Listing
Modified Thu Feb 11 08:09:23 2010 UTC (10 years, 6 months ago) by egils
Improved error reporting. Bugfixes

Revision 534 - Directory Listing
Modified Tue Feb 9 14:13:01 2010 UTC (10 years, 6 months ago) by heikok
allowing for filename[] arrays

Revision 533 - Directory Listing
Modified Mon Feb 8 13:11:35 2010 UTC (10 years, 6 months ago) by egils
Minor bugfixes

Revision 532 - Directory Listing
Modified Mon Feb 8 10:41:09 2010 UTC (10 years, 6 months ago) by egils
Minor bugfixes

Revision 530 - Directory Listing
Modified Fri Feb 5 13:04:40 2010 UTC (10 years, 6 months ago) by egils
New functionality: Only register new files already in repository with URL using
newfiles.php (required by OSISAF). These files need not comply with the 
dataset naming convention.

Revision 494 - Directory Listing
Modified Fri Jan 15 14:14:32 2010 UTC (10 years, 6 months ago) by egils
Made debug available

Revision 492 - Directory Listing
Modified Fri Jan 15 13:07:36 2010 UTC (10 years, 6 months ago) by heikok
final test before implementation of PostGIS

Revision 490 - Directory Listing
Modified Tue Jan 12 14:55:34 2010 UTC (10 years, 7 months ago) by heikok
adding new functionality to NcFind
extending ncExtractLatLon to detect polygons

Revision 487 - Directory Listing
Modified Thu Jan 7 15:28:29 2010 UTC (10 years, 7 months ago) by heikok
indentation fixes, smaller fixes while testing ncExtractLatLong.pl

Revision 478 - Directory Listing
Modified Wed Jan 6 14:14:19 2010 UTC (10 years, 7 months ago) by egils
Removed not-intended decode from UTF-8 of user-supplied strings

Revision 476 - Directory Listing
Modified Wed Jan 6 11:32:02 2010 UTC (10 years, 7 months ago) by heikok
moving ncfind to MetNo::NcFind
adding documentation and tests to MetNo::NcFind
adding ncExtractLatLong test-version

Revision 472 - Directory Listing
Modified Wed Dec 16 13:54:37 2009 UTC (10 years, 7 months ago) by egils
Fixed test for existing directory key

Revision 471 - Directory Listing
Modified Wed Dec 16 08:52:04 2009 UTC (10 years, 7 months ago) by egils
Fixed href attribute

Revision 466 - Directory Listing
Modified Mon Nov 30 09:01:52 2009 UTC (10 years, 8 months ago) by egils
Ported revisions 463-465 from branches/2.3

Revision 439 - Directory Listing
Modified Thu Oct 22 13:26:21 2009 UTC (10 years, 9 months ago) by heikok
adding map and lat/lon layer

Revision 433 - Directory Listing
Modified Thu Oct 22 08:58:15 2009 UTC (10 years, 9 months ago) by heikok
Making WMS_ONLINE_RESOURCE a configurational parameter,
fixing singleTile and units in WMC document example

Revision 424 - Directory Listing
Modified Thu Oct 8 10:51:02 2009 UTC (10 years, 10 months ago) by heikok
removing idea of using hard-links instead of copy. Hard-links might be modified in the original place by i.e. cp, ftp or http and cause trouble with the linked file

Revision 423 - Directory Listing
Modified Wed Oct 7 11:05:02 2009 UTC (10 years, 10 months ago) by heikok
using internal 'head' command

Revision 422 - Directory Listing
Modified Wed Oct 7 10:55:44 2009 UTC (10 years, 10 months ago) by heikok
using memory efficient version of remove_cr_from_file in upload_monitor
mofing that function to Utils for easier unit-tests

Revision 419 - Directory Listing
Modified Fri Sep 25 10:20:22 2009 UTC (10 years, 10 months ago) by heikok
adding namespace-prefix to example

Revision 414 - Directory Listing
Modified Wed Aug 26 13:59:36 2009 UTC (10 years, 11 months ago) by heikok
checking for well-formed xml for wms-info
checking for validating xml for wms-info and projection-info
adding schemas for wmcSetup and fimexProjections, wmcSetup currently weak-check, waiting on feedback from Trond according context.xsd from OGC

Revision 404 - Directory Listing
Modified Sat Aug 22 12:02:53 2009 UTC (10 years, 11 months ago) by heikok
visualizing of WMS in the search-table
Removing WMS and FIMEX config buttons if WMS_URL or FIMEX_PROGRAM not set in master_config.txt
configuring example with WMS and FIMEX_PROGRAM

Revision 403 - Directory Listing
Modified Sat Aug 22 11:18:41 2009 UTC (10 years, 11 months ago) by heikok
retrieval of WMC from database via getWMC.php page + mmWMS.inc
change of namespaces for wmsEdit

Revision 401 - Directory Listing
Modified Fri Aug 21 11:58:55 2009 UTC (10 years, 11 months ago) by heikok
enabling editing of WMS setup

Revision 393 - Directory Listing
Modified Wed Jul 22 10:33:10 2009 UTC (11 years ago) by heikok
changing doc

Revision 392 - Directory Listing
Modified Wed Jul 22 10:26:12 2009 UTC (11 years ago) by heikok
fixing bug in mmFimex.inc for property with string-value false rathern than boolean false
several fixes on projection parameters

Revision 390 - Directory Listing
Modified Wed Jul 22 07:46:33 2009 UTC (11 years ago) by heikok
enabling full download of reprojected nc-files

Revision 385 - Directory Listing
Modified Mon Jul 20 13:34:20 2009 UTC (11 years ago) by heikok
adding projectionInfo addition page to Administration in upload

Revision 375 - Directory Listing
Modified Thu Jul 16 08:36:05 2009 UTC (11 years, 1 month ago) by egils
Ported r372 from branches/2.2 (editing directory keys)

Revision 373 - Directory Listing
Modified Thu Jul 16 08:29:48 2009 UTC (11 years, 1 month ago) by egils
Ported r349 from branches/2.2 (Removed die when unable to stat ftp-uploaded file)

Revision 347 - Directory Listing
Modified Tue Jun 16 08:54:36 2009 UTC (11 years, 1 month ago) by egils
Ported r345 from branches/2.2 (IDN_Node, show_users.php)

Revision 328 - Directory Listing
Modified Mon May 11 15:50:46 2009 UTC (11 years, 3 months ago) by heikok
switching from start_services.sh and stop_services.sh to lsb-conform startup-script metamodInit.sh

Revision 326 - Directory Listing
Modified Mon May 11 08:08:21 2009 UTC (11 years, 3 months ago) by heikok
removing problems in character-stes in 04_Dataset.t, smaller fixes in php code

Revision 325 - Directory Listing
Modified Mon May 11 06:55:35 2009 UTC (11 years, 3 months ago) by heikok
everything except shell-scripts and static files made run-time configurable

Revision 324 - Directory Listing
Modified Mon May 11 06:20:37 2009 UTC (11 years, 3 months ago) by heikok
making upl runtime configurable

Revision 309 - Directory Listing
Modified Wed May 6 13:34:34 2009 UTC (11 years, 3 months ago) by heikok
removing all config statements from perl-files, except in use lib ...;

Revision 300 - Directory Listing
Modified Thu Apr 16 12:13:10 2009 UTC (11 years, 3 months ago) by egils
Fixed bug in getFiletype (tar not anchored at end of string).
Reverted link_or_copy to only copy.

Revision 295 - Directory Listing
Modified Thu Apr 2 09:45:52 2009 UTC (11 years, 4 months ago) by egils
Fixed bug regarding datacollection_period (from/to): In nc-files
this information is encoded as global attributes 'start_date' and 'stop_date'. When applying rules like
'take_lowest_date', only dates in the new nc-files were considered. Dates in the existing XML file were
not considered since they had wrong names (datacollection_period*, not start/stop_date).

Revision 291 - Directory Listing
Modified Tue Mar 31 13:56:20 2009 UTC (11 years, 4 months ago) by egils
Added possibility to upload files to a dataset by the METAMOD operator, without sending the user in an E-mail in case of errors (the E-mail will be sent to the operator instead)

Revision 290 - Directory Listing
Modified Mon Mar 30 09:35:51 2009 UTC (11 years, 4 months ago) by egils
Improved feedback to user when testing more than one file,
better clean up in etaf directory.

Revision 289 - Directory Listing
Modified Mon Mar 30 08:58:21 2009 UTC (11 years, 4 months ago) by egils
Fixed bug in getFiletype (file type for tar files)

Revision 288 - Directory Listing
Modified Mon Mar 30 07:54:02 2009 UTC (11 years, 4 months ago) by egils
Added possibility to run upload_monitor.pl with one
argument = 'newday' which will do one iteration of the main loop, and this iteration will activate
the clean up routines normally done only one time each day

Revision 287 - Directory Listing
Modified Fri Mar 27 12:14:32 2009 UTC (11 years, 4 months ago) by egils
When running digest_nc.pl on each file to create child xml files,
take file names from @uploaded_files and not @digest_input (which may contain files already in repository).

Revision 286 - Directory Listing
Modified Fri Mar 27 10:40:12 2009 UTC (11 years, 4 months ago) by egils
Reverted change where @file_content_arr were getting newlines at the end of each array element

Revision 284 - Directory Listing
Modified Wed Mar 25 11:23:55 2009 UTC (11 years, 4 months ago) by egils
Added possibility to read the list of parent files from a text file

Revision 282 - Directory Listing
Modified Tue Mar 24 12:54:55 2009 UTC (11 years, 4 months ago) by egils
Ignore ownertags in file level datasets when searching.
Improved listing of existing directory datasets in upload web interface.

Revision 279 - Directory Listing
Modified Thu Mar 19 18:34:10 2009 UTC (11 years, 4 months ago) by heikok
change drpath to match new thredds config: dataset=institution/project/file.nc for all datasets

Revision 276 - Directory Listing
Modified Tue Mar 17 13:10:12 2009 UTC (11 years, 4 months ago) by heikok
change path for 2nd level opendap-path
don't recreate existing datasets

Revision 275 - Directory Listing
Modified Tue Mar 17 10:45:59 2009 UTC (11 years, 4 months ago) by heikok
change path for 2nd level opendap-path

Revision 272 - Directory Listing
Modified Thu Mar 12 11:57:01 2009 UTC (11 years, 5 months ago) by heikok
remove doublecheck of getDatatype (#270), test identical to #271 and therefore no longer needed.

Revision 271 - Directory Listing
Modified Thu Mar 12 11:47:25 2009 UTC (11 years, 5 months ago) by heikok
using own filetype detection instead of using file/magic

Revision 270 - Directory Listing
Modified Thu Mar 12 10:40:05 2009 UTC (11 years, 5 months ago) by egils
Ported r267 and r268 from branches/2.1

Revision 258 - Directory Listing
Modified Mon Mar 9 12:21:36 2009 UTC (11 years, 5 months ago) by egils
fixing default dataset-name 2nd try

Revision 257 - Directory Listing
Modified Mon Mar 9 12:06:51 2009 UTC (11 years, 5 months ago) by egils
fixing default dataset-name

Revision 253 - Directory Listing
Modified Tue Mar 3 14:13:09 2009 UTC (11 years, 5 months ago) by heikok
fixing type File->Spec -> File::Spec

Revision 252 - Directory Listing
Modified Tue Mar 3 13:03:22 2009 UTC (11 years, 5 months ago) by heikok
fixing precompilation of patterns

Revision 244 - Directory Listing
Modified Wed Feb 25 15:07:33 2009 UTC (11 years, 5 months ago) by egils
Ported bugfix from branches/2.1

Revision 236 - Directory Listing
Modified Mon Feb 23 16:13:18 2009 UTC (11 years, 5 months ago) by heikok
tested and fixed chreateChildDatasets.pl for production use

Revision 234 - Directory Listing
Modified Mon Feb 23 14:24:59 2009 UTC (11 years, 5 months ago) by heikok
create file to generate child datasets from already existing files

Revision 231 - Directory Listing
Modified Mon Feb 23 09:55:59 2009 UTC (11 years, 5 months ago) by heikok
changing parameters of Utils.pm to make it more general usable (all type of file-checks, not only filename-pattern), using changes in upload_monitor and import_dataset
removing touch from import_dataset, cleanup code

Revision 230 - Directory Listing
Modified Fri Feb 20 21:06:04 2009 UTC (11 years, 5 months ago) by heikok
getting import_dataset.pl to work with 2 level xml-files
import_dataset.pl runs now also on dirs, used for testing and in create_and_load_all.sh
digest_nc.pl generates now correct dataset-names for parent and child files, requires new command-line flag for children


Revision 227 - Directory Listing
Modified Fri Feb 20 13:26:25 2009 UTC (11 years, 5 months ago) by heikok
moving findFiles to Metamod::Utils

Revision 226 - Directory Listing
Modified Fri Feb 20 12:49:06 2009 UTC (11 years, 5 months ago) by heikok
generation of 2nd level xml files
using File::Spec for creation of pathes

Revision 224 - Directory Listing
Modified Fri Feb 20 10:36:42 2009 UTC (11 years, 5 months ago) by heikok
move find, rm, remove_cr.sh to perl-interna
use File::Spec for operations on file-pathes

Revision 207 - Directory Listing
Modified Thu Jan 29 13:08:41 2009 UTC (11 years, 6 months ago) by heikok
testing of better utf-8 support, no longer writing warnings by default if no utf-8 data is send in ($DEBUG in ForeignDataset)

Revision 205 - Directory Listing
Modified Wed Jan 28 14:17:04 2009 UTC (11 years, 6 months ago) by heikok
better detection of encodings, no longer possible to generate xml-files with wrong encoding. Warning when trying. Not tested yet.

Revision 195 - Directory Listing
Modified Fri Jan 23 13:25:37 2009 UTC (11 years, 6 months ago) by egils
Made it impossible to create a directory owned by another user

Revision 187 - Directory Listing
Modified Thu Jan 22 12:40:53 2009 UTC (11 years, 6 months ago) by egils
Corrections in DIF template, small bugfixes

Revision 185 - Directory Listing
Modified Mon Jan 19 11:56:13 2009 UTC (11 years, 6 months ago) by egils
Improved robustness and error messages

Revision 183 - Directory Listing
Modified Fri Jan 16 13:36:38 2009 UTC (11 years, 6 months ago) by egils
Disabled sending E-mails to users under test runs

Revision 181 - Directory Listing
Modified Tue Jan 13 08:59:56 2009 UTC (11 years, 7 months ago) by egils
Added new web page for testing a netCDF file without uploading.
Small adjustments in harvester.pl.'

Revision 173 - Directory Listing
Modified Thu Jan 8 12:45:55 2009 UTC (11 years, 7 months ago) by egils
Improved file checking and error reporting

Revision 167 - Directory Listing
Modified Thu Dec 18 13:07:32 2008 UTC (11 years, 7 months ago) by egils
Preliminary hack: Made it possible to change the ownertag
of a dataset by editing the corresponding *.xmd file. This is achieved by making digest_nc.pl
keep the ownertag of an existing dataset, and by removing a check in import_dataset.pl that no
dataset with same name but different ownertag are imported. Datasets with same name will now
overwrite existing dataset unconditionally. This hack is needed to make it possible to tidy up
datasets on the damocles installation.

Revision 160 - Directory Listing
Modified Fri Dec 12 12:07:52 2008 UTC (11 years, 8 months ago) by egils
Added missing statement for setting the list of repository
files that should be reprocessed

Revision 150 - Directory Listing
Modified Mon Dec 8 14:22:51 2008 UTC (11 years, 8 months ago) by egils
Removed multiple writes of same <file> element to userfile

Revision 147 - Directory Listing
Modified Mon Dec 8 10:27:42 2008 UTC (11 years, 8 months ago) by egils
Added mmTtime class for both perl and PHP

Revision 139 - Directory Listing
Modified Tue Dec 2 12:00:44 2008 UTC (11 years, 8 months ago) by egils
Removed possibility for setting an undef into array referenced from the %metadata hash

Revision 136 - Directory Listing
Modified Mon Dec 1 14:32:51 2008 UTC (11 years, 8 months ago) by egils
Reverted sending test output to STDERR

Revision 125 - Directory Listing
Modified Tue Nov 25 11:39:18 2008 UTC (11 years, 8 months ago) by heikok
switching from easternmost_longitude, westernmost_longitude, northernmost_latitude, southernmost_latitude to bounding_box

Revision 112 - Directory Listing
Modified Thu Nov 20 09:03:53 2008 UTC (11 years, 8 months ago) by egils
Ported bugfixes in r96 from branches/2.0

Revision 91 - Directory Listing
Modified Fri Nov 7 13:28:07 2008 UTC (11 years, 9 months ago) by heikok
using Metamod::Dataset to read/write datasets in digest_nc.pl
fixing default creationDate representation in mmDataset.inc and Dataset.pm
fixing namespace-bugs and metadata name-bugs in oldDataset2MM2.xslt
adding tests to detect those bugs

Revision 87 - Directory Listing
Modified Fri Nov 7 10:38:06 2008 UTC (11 years, 9 months ago) by egils
Common syserrors file for upload and harvest.

Revision 86 - Directory Listing
Modified Fri Nov 7 08:11:52 2008 UTC (11 years, 9 months ago) by egils
New directory path for quadtreeuse.pm. Bugfixes

Revision 72 - Directory Listing
Modified Mon Nov 3 12:22:05 2008 UTC (11 years, 9 months ago) by egils
Moved perl module quadtreeuse.pm to common

Revision 65 - Directory Listing
Modified Wed Oct 29 11:14:43 2008 UTC (11 years, 9 months ago) by heikok
reading dataset and dataset2, writing dataset2 

Revision 55 - Directory Listing
Modified Mon Oct 20 13:05:40 2008 UTC (11 years, 9 months ago) by heikok
no longer using 'timesync' dataset synchronisation (switching to flock based sync)

Revision 54 - Directory Listing
Modified Mon Oct 20 07:44:02 2008 UTC (11 years, 9 months ago) by heikok
fixes from Anders Hartman on example, usage with postgres 8.3?, more general and robust php-code 

Revision 53 - Directory Listing
Modified Fri Oct 17 12:05:08 2008 UTC (11 years, 9 months ago) by heikok
enabling flock of datasets for reading/writing in import_dataset.pl, readDataset.inc, digest_nc.pl and quest.php

Revision 39 - Directory Listing
Modified Fri Oct 3 09:59:17 2008 UTC (11 years, 10 months ago) by heikok
connection quest and upload to add metadata per directory

Revision 37 - Directory Listing
Modified Fri Sep 26 12:18:42 2008 UTC (11 years, 10 months ago) by heikok
adding xsd schema for conf_digest_nc, extracted from xml file

Revision 33 - Directory Listing
Modified Thu Sep 18 08:43:34 2008 UTC (11 years, 10 months ago) by egils
Ported r31 (ftp_events file) from branches/2.0

Revision 30 - Directory Listing
Modified Mon Sep 15 09:50:54 2008 UTC (11 years, 11 months ago) by egils
Ported r28 (revert XML history) to trunk

Revision 25 - Directory Listing
Modified Mon Sep 1 13:16:57 2008 UTC (11 years, 11 months ago) by egils
Ported r24 (CRLF endings in CDL files) from branches/2.0

Revision 11 - Directory Listing
Modified Mon Jul 21 12:52:33 2008 UTC (12 years ago) by egils
Added removal of ftp uploaded files after N days

Revision 9 - Directory Listing
Modified Fri Jul 18 10:05:34 2008 UTC (12 years ago) by egils
Simplified problem files reporting

Revision 8 - Directory Listing
Modified Fri Jul 18 08:32:03 2008 UTC (12 years ago) by egils
Another small bug fix

Revision 7 - Directory Listing
Modified Thu Jul 17 14:02:57 2008 UTC (12 years ago) by egils
Small bug fix

Revision 6 - Directory Listing
Modified Wed Jul 16 12:39:26 2008 UTC (12 years, 1 month ago) by egils
Improved robustness. Waiting 5 hours before moving unknown uploaded files to the problemfiles directory

Revision 5 - Directory Listing
Modified Fri Jun 27 11:20:06 2008 UTC (12 years, 1 month ago) by egils
Bug fixes

Revision 4 - Directory Listing
Modified Thu Jun 26 08:11:40 2008 UTC (12 years, 1 month ago) by egils
Improved documentation, removed file

Revision 2 - Directory Listing
Added Mon Jun 23 12:30:38 2008 UTC (12 years, 1 month ago) by egils
Initial import

svn@met.no
ViewVC Help
Powered by ViewVC 1.1.5