Home › Forums › implementation › Implementation discussion ›

Kettle Transforms

Submitted by CWearne on Thu, 12/08/2021 - 15:58
in

Dear Forum Members,

After reading through the appropriate forum discussions I realise this is a familiar  and no doubt frustrating question/topic. I am exploring openvpms and thus far have found both the implementation and use to be quite straight forward. I am at the point of wanting to move across some data from my current PMS to see how feasible the conversion is.

I was wondering if someone might be kind enough to provide me with some kettle transform files that I can use as a template? I have kettle set up to the point the openvpms plugin is registered and there is a database connection that can see the openvpms mysql database. My current PMS makes the export of csv files easy so my plan unless someone firmly convinces me otherwise was to use the csv files as the source files.

Any help very much appreciated.

Regards

Chris

‹ Moving products to new preferred supplier Kettle Transforms ›

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Kettle Transforms

Submitted by CWearne on Fri, 13/08/2021 - 02:43.

Further to my question above, I have had a little more of a play with kettle and realise now the plugin is not working correctly. On attempting to use the opvpms plugin I receive in the logging:

org.openvpms.etl.pentaho.LoaderPluginMeta@5a4f149e - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Failed to load application context

From reading through historical posts I realise this can be an issue with the version of spring used in PDI vs the plugin. Any guidance on resolving this much appreciated.

Kind regards

Chris

Re: Kettle Transforms

Submitted by tanderson on Fri, 13/08/2021 - 10:30.

You might get that error if your database connection isn't set up correctly.

If you open your OpenVPMS Loader step and click Edit next on the Connection field, it should look something like the below. Click Test to ensure you can connect to the database.

If that works, but the plugin still won't load, see if you find more information in the logs in <PDI_HOME>/logs/

With regards to migration, this is a non-trivial task; you need a good understanding of the data model of both the source PMS and OpenVPMS. E.g the following are (old) transforms for RxWorks customers, patients and invoice items.

 

 

Re: Kettle Transforms

Submitted by CWearne on Fri, 13/08/2021 - 16:22.

Many  thanks for your reply.

The database connection appears fine as I can use the explorer to visualise the tables. Attached are the screen shots from the explorer and connection test.

The logs aren't very helpful I'm afraid. They contain the line:

2021-08-13 07:10:18.096 ERROR <init of OpenVPMS Loader.0 (Thread-77)> [/home/chris/pentaho/ClientTransform.ktr]  Error initializing step [OpenVPMS Loader]

Ever so slightly more useful is the error output from the dummy run of the transform process itself:

2021/08/13 07:10:18 - org.openvpms.etl.pentaho.LoaderPluginMeta@66f8b0c - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Failed to load application context

 

With regards the old PMS schema, I designed it so I have a fairly good knowledge of its structure. Its in a standard normalised relational form. I'm not used to this non-normalised entity based system used with openvpms but hope to get my head around it.

 

Kind regards

Chris

AttachmentSize
Screenshot from 2021-08-13 07-05-41.png 77.41 KB
Screenshot from 2021-08-13 07-07-14.png 150.2 KB

Re: Kettle Transforms

Submitted by tanderson on Fri, 13/08/2021 - 16:40.

I would expect to see a stack trace indicating the cause of the failure in spoon.log.
It should also been shown in the Logging tab.

E.g.:

2021/08/13 16:34:20 - Spoon - Running transformation using the Kettle execution engine
2021/08/13 16:34:20 - Spoon - Transformation opened.
2021/08/13 16:34:20 - Spoon - Launching transformation [test2]...
2021/08/13 16:34:20 - Spoon - Started the transformation execution.
2021/08/13 16:34:20 - test2 - Dispatching started for transformation [test2]
2021/08/13 16:34:21 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - Initialising application context, using database URL: jdbc:mysql://localhost:3306/openvpms_dev
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Failed to load application context
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imobjectInterceptor' defined in class path resource [applicationContext.xml]: Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [applicationContext.xml]: Invocation of init method failed; nested exception is org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:342)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1699)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1444)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.openvpms.etl.pentaho.ApplicationContextMgr.load(ApplicationContextMgr.java:112)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.openvpms.etl.pentaho.ApplicationContextMgr.getContext(ApplicationContextMgr.java:77)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.openvpms.etl.pentaho.LoaderPluginMeta.getContext(LoaderPluginMeta.java:405)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.openvpms.etl.pentaho.LoaderPluginMeta.getStep(LoaderPluginMeta.java:361)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1041)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.pentaho.di.ui.spoon.trans.TransGraph$30.run(TransGraph.java:4281)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at java.lang.Thread.run(Thread.java:748)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [applicationContext.xml]: Invocation of init method failed; nested exception is org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     ... 19 more
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:275)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:179)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:119)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.build(MetadataBuildingProcess.java:84)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:474)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:85)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:689)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:724)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.orm.hibernate5.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:616)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.orm.hibernate5.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:600)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     ... 26 more
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 - Caused by: org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:100)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:137)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:94)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
2021/08/13 16:34:22 - org.openvpms.etl.pentaho.LoaderPluginMeta@1ce60c43 -     ... 43 more
2021/08/13 16:34:22 - OpenVPMS Loader.0 - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Error initializing step [OpenVPMS Loader]
2021/08/13 16:34:22 - test2 - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Step [OpenVPMS Loader.0] failed to initialize!
2021/08/13 16:34:22 - Spoon - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : test2: preparing transformation execution failed
2021/08/13 16:34:22 - Spoon - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : org.pentaho.di.core.exception.KettleException:
2021/08/13 16:34:22 - Spoon - We failed to initialize at least one step.  Execution can not begin!
2021/08/13 16:34:22 - Spoon -
2021/08/13 16:34:22 - Spoon -
2021/08/13 16:34:22 - Spoon -     at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1286)
2021/08/13 16:34:22 - Spoon -     at org.pentaho.di.ui.spoon.trans.TransGraph$30.run(TransGraph.java:4281)
2021/08/13 16:34:22 - Spoon -     at java.lang.Thread.run(Thread.java:748)

Re: Kettle Transforms

Submitted by tanderson on Fri, 13/08/2021 - 17:33.

Also, if you end up not wanting to do the migration yourself, send a mail to support[at]openvpms[dot]com and we can give you a quote.

Re: Kettle Transforms

Submitted by CWearne on Fri, 13/08/2021 - 18:43.

Please see the attached output from the spoon.log file: Aug 13, 2021 7:01:09 AM

org.apache.karaf.main.Main$KarafLockCallback lockAcquired INFO: Lock acquired. Setting startlevel to 100 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Examining bundle for SPI provider: org.eclipse.jetty.http Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://113.0:0/META-INF/services/org.eclipse.jetty.http.HttpFieldPreEncoder Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.http.Http1FieldPreEncoder Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@51b7ab6d Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.http.HttpFieldPreEncoder in bundle org.eclipse.jetty.http Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Examining bundle for SPI provider: org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://130.0:0/META-INF/services/org.eclipse.jetty.websocket.api.extensions.Extension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.common.extensions.identity.IdentityExtension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@67c85541 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.websocket.api.extensions.Extension in bundle org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.common.extensions.fragment.FragmentExtension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@3a722390 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.websocket.api.extensions.Extension in bundle org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@70440291 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.websocket.api.extensions.Extension in bundle org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.common.extensions.compress.DeflateFrameExtension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@58c3b19d Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.websocket.api.extensions.Extension in bundle org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.common.extensions.compress.XWebkitDeflateFrameExtension Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@52f02186 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: org.eclipse.jetty.websocket.api.extensions.Extension in bundle org.eclipse.jetty.websocket.common Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Examining bundle for SPI provider: org.eclipse.jetty.websocket.javax.websocket Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://131.0:0/META-INF/services/javax.websocket.ContainerProvider Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.jsr356.JettyClientContainerProvider Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@2fd1ca04 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: javax.websocket.ContainerProvider in bundle org.eclipse.jetty.websocket.javax.websocket Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Examining bundle for SPI provider: org.eclipse.jetty.websocket.javax.websocket.server Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://132.0:0/META-INF/services/javax.servlet.ServletContainerInitializer Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@f60d50b Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: javax.servlet.ServletContainerInitializer in bundle org.eclipse.jetty.websocket.javax.websocket.server Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://132.0:0/META-INF/services/javax.websocket.server.ServerEndpointConfig$Configurator Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.jsr356.server.ContainerDefaultConfigurator Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@3755381e Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: javax.websocket.server.ServerEndpointConfig$Configurator in bundle org.eclipse.jetty.websocket.javax.websocket.server Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Examining bundle for SPI provider: org.eclipse.jetty.websocket.server Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Found SPI resource: bundle://133.0:0/META-INF/services/javax.servlet.ServletContainerInitializer Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Loaded SPI provider: class org.eclipse.jetty.websocket.server.NativeWebSocketServletContainerInitializer Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered service: org.apache.felix.framework.ServiceRegistrationImpl@74c73a65 Aug 13, 2021 7:01:12 AM org.apache.aries.spifly.BaseActivator log INFO: Registered provider: javax.servlet.ServletContainerInitializer in bundle org.eclipse.jetty.websocket.server 2021-08-13 07:01:12.504:INFO::FelixStartLevel: Logging initialized @10812ms to org.eclipse.jetty.util.log.StdErrLog Aug 13, 2021 7:01:12 AM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-management (157) [org.apache.cxf.management.InstrumentationManager] Aug 13, 2021 7:01:12 AM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-rs-service-description (162) [org.apache.cxf.jaxrs.model.wadl.WadlGenerator] Aug 13, 2021 7:01:12 AM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-transports-http (164) [org.apache.cxf.transport.http.HTTPTransportFactory, org.apache.cxf.transport.http.HTTPWSDLExtensionLoader, org.apache.cxf.transport.http.policy.HTTPClientAssertionBuilder, org.apache.cxf.transport.http.policy.HTTPServerAssertionBuilder, org.apache.cxf.transport.http.policy.NoOpPolicyInterceptorProvider] Aug 13, 2021 7:01:12 AM org.apache.cxf.transport.http.osgi.ServletExporter updated INFO: Registering new instance of "/cxf" servlet 2021-08-13 07:01:13.395:INFO:oejws.WebSocketServerFactory:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:13.503:INFO:oejs.session:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): DefaultSessionIdManager workerName=node0 2021-08-13 07:01:13.503:INFO:oejs.session:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): No SessionScavenger set, using defaults 2021-08-13 07:01:13.505:INFO:oejs.session:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): node0 Scavenging every 600000ms 2021-08-13 07:01:13.534:INFO:oejsh.ContextHandler:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=org.apache.cxf.cxf-rt-transports-http [164], contextID=default]} 2021-08-13 07:01:13.539:INFO:oejs.Server:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): jetty-9.4.18.v20190429; built: 2019-04-29T20:42:08.989Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_292-8u292-b10-0ubuntu1~18.04-b10 2021-08-13 07:01:13.588:INFO:oejs.AbstractConnector:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): Started default@51f53fb6{HTTP/1.1,[http/1.1]}{0.0.0.0:9051} 2021-08-13 07:01:13.589:INFO:oejs.Server:CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]): Started @11897ms Aug 13, 2021 7:01:13 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /i18n Aug 13, 2021 7:01:13 AM org.pentaho.caching.impl.PentahoCacheManagerFactory$RegistrationHandler$1 onSuccess INFO: New Caching Service registered 2021-08-13 07:01:13.789:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:13.792:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-i18n-webservice-bundle [176], contextID=default]} 2021-08-13 07:01:14.584:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:14.588:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-pdi-platform [250], contextID=default]} 2021-08-13 07:01:15.407:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.429:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=core-ui [266], contextID=default]} 2021-08-13 07:01:15.465:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.468:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=org.pentaho.requirejs-manager-impl [267], contextID=default]} 2021-08-13 07:01:15.484:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.486:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular [268], contextID=default]} 2021-08-13 07:01:15.495:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.497:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-animate [269], contextID=default]} 2021-08-13 07:01:15.514:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.519:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-i18n [270], contextID=default]} 2021-08-13 07:01:15.530:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.532:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-route [271], contextID=default]} 2021-08-13 07:01:15.541:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.543:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-sanitize [272], contextID=default]} 2021-08-13 07:01:15.550:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.553:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-translate [273], contextID=default]} 2021-08-13 07:01:15.562:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.565:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-ui-router [274], contextID=default]} 2021-08-13 07:01:15.574:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.577:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-dojo [275], contextID=default]} 2021-08-13 07:01:15.586:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.589:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-require-css [276], contextID=default]} 2021-08-13 07:01:15.597:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.599:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-requirejs-text [277], contextID=default]} Aug 13, 2021 7:01:15 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /repositories Aug 13, 2021 7:01:15 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /browser 2021-08-13 07:01:15.898:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.903:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=repositories-plugin-core [278], contextID=default]} 2021-08-13 07:01:15.903:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:15.906:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-i18n [281], contextID=default]} Aug 13, 2021 7:01:15 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /browser-new Aug 13, 2021 7:01:16 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /get-fields 2021-08-13 07:01:16.043:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.046:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=file-open-save-core [280], contextID=default]} Aug 13, 2021 7:01:16 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /connection 2021-08-13 07:01:16.287:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.293:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=file-open-save-new-core [283], contextID=default]} 2021-08-13 07:01:16.403:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.406:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=get-fields-core [284], contextID=default]} Aug 13, 2021 7:01:16 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /marketplace 2021-08-13 07:01:16.688:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.689:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.691:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-angular-ui-bootstrap-bower [287], contextID=default]} 2021-08-13 07:01:16.695:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=connections-ui [285], contextID=default]} 2021-08-13 07:01:16.698:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.701:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-jquery [288], contextID=default]} 2021-08-13 07:01:16.712:INFO:oejws.WebSocketServerFactory:FelixStartLevel: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.714:INFO:oejsh.ContextHandler:FelixStartLevel: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-webjars-underscore [289], contextID=default]} 2021-08-13 07:01:16.863:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:16.869:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=pentaho-marketplace-di [286], contextID=default]} Aug 13, 2021 7:01:16 AM org.apache.cxf.endpoint.ServerImpl initDestination INFO: Setting the server's publish address to be /hadoop-cluster 2021-08-13 07:01:18.160:INFO:oejws.WebSocketServerFactory:FelixDispatchQueue: No DecoratedObjectFactory provided, using new org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1] 2021-08-13 07:01:18.165:INFO:oejsh.ContextHandler:FelixDispatchQueue: Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=hadoop-cluster-ui [291], contextID=default]} Graphics2D from BufferedImage lacks BUFFERED_IMAGE hint 2021/08/13 07:01:21 - Carte - Installing timer to purge stale objects after 1440 minutes. Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2021/08/13 07:10:17 - Spoon - Running transformation using the Kettle execution engine 2021/08/13 07:10:17 - ClientTransform - Transformation is pre-loaded. 2021/08/13 07:10:17 - ClientTransform - nr of steps to run : 3 , nr of hops : 1 2021/08/13 07:10:17 - Spoon - Transformation opened. 2021/08/13 07:10:17 - Spoon - Launching transformation [ClientTransform]... 2021/08/13 07:10:17 - Spoon - Started the transformation execution. 2021/08/13 07:10:18 - ClientTransform - Dispatching started for transformation [ClientTransform] 2021/08/13 07:10:18 - ClientTransform - Nr of arguments detected:0 2021/08/13 07:10:18 - ClientTransform - This is not a replay transformation 2021/08/13 07:10:18 - ClientTransform - I found 3 different steps to launch. 2021/08/13 07:10:18 - ClientTransform - Allocating rowsets... 2021/08/13 07:10:18 - ClientTransform - Allocating rowsets for step 0 --> CSV file input 2021/08/13 07:10:18 - ClientTransform - prevcopies = 1, nextcopies=1 2021/08/13 07:10:18 - ClientTransform - Transformation allocated new rowset [CSV file input.0 - OpenVPMS Loader.0] 2021/08/13 07:10:18 - ClientTransform - Allocated 1 rowsets for step 0 --> CSV file input 2021/08/13 07:10:18 - ClientTransform - Allocating rowsets for step 1 --> OpenVPMS Loader 2021/08/13 07:10:18 - ClientTransform - Allocated 1 rowsets for step 1 --> OpenVPMS Loader 2021/08/13 07:10:18 - ClientTransform - Allocating rowsets for step 2 --> XML output 2021/08/13 07:10:18 - ClientTransform - Allocated 1 rowsets for step 2 --> XML output 2021/08/13 07:10:18 - ClientTransform - Allocating Steps & StepData... 2021/08/13 07:10:18 - ClientTransform - Transformation is about to allocate step [CSV file input] of type [CsvInput] 2021/08/13 07:10:18 - ClientTransform - Step has nrcopies=1 2021/08/13 07:10:18 - CSV file input.0 - distribution activated 2021/08/13 07:10:18 - CSV file input.0 - Starting allocation of buffers & new threads... 2021/08/13 07:10:18 - CSV file input.0 - Step info: nrinput=0 nroutput=1 2021/08/13 07:10:18 - CSV file input.0 - output rel. is 1:1 2021/08/13 07:10:18 - CSV file input.0 - Found output rowset [CSV file input.0 - OpenVPMS Loader.0] 2021/08/13 07:10:18 - CSV file input.0 - Finished dispatching 2021/08/13 07:10:18 - ClientTransform - Transformation has allocated a new step: [CSV file input].0 2021/08/13 07:10:18 - ClientTransform - Transformation is about to allocate step [OpenVPMS Loader] of type [OpenVPMSLoaderPlugin] 2021/08/13 07:10:18 - ClientTransform - Step has nrcopies=1 2021/08/13 07:10:18 - org.openvpms.etl.pentaho.LoaderPluginMeta@66f8b0c - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Failed to load application context 2021/08/13 07:10:18 - OpenVPMS Loader.0 - distribution activated 2021/08/13 07:10:18 - OpenVPMS Loader.0 - Starting allocation of buffers & new threads... 2021/08/13 07:10:18 - OpenVPMS Loader.0 - Step info: nrinput=1 nroutput=0 2021/08/13 07:10:18 - OpenVPMS Loader.0 - Got previous step from [OpenVPMS Loader] #0 --> CSV file input 2021/08/13 07:10:18 - OpenVPMS Loader.0 - input rel is 1:1 2021/08/13 07:10:18 - OpenVPMS Loader.0 - Found input rowset [CSV file input.0 - OpenVPMS Loader.0] 2021/08/13 07:10:18 - OpenVPMS Loader.0 - Finished dispatching 2021/08/13 07:10:18 - ClientTransform - Transformation has allocated a new step: [OpenVPMS Loader].0 2021/08/13 07:10:18 - ClientTransform - Transformation is about to allocate step [XML output] of type [XMLOutput] 2021/08/13 07:10:18 - ClientTransform - Step has nrcopies=1 2021/08/13 07:10:18 - XML output.0 - distribution activated 2021/08/13 07:10:18 - XML output.0 - Starting allocation of buffers & new threads... 2021/08/13 07:10:18 - XML output.0 - Step info: nrinput=0 nroutput=0 2021/08/13 07:10:18 - XML output.0 - Finished dispatching 2021/08/13 07:10:18 - ClientTransform - Transformation has allocated a new step: [XML output].0 2021/08/13 07:10:18 - ClientTransform - This transformation can be replayed with replay date: 2021/08/13 07:10:18 2021/08/13 07:10:18 - ClientTransform - Initialising 3 steps... 2021/08/13 07:10:18 - CSV file input.0 - Released server socket on port 0 2021/08/13 07:10:18 - XML output.0 - Released server socket on port 0 2021/08/13 07:10:18 - OpenVPMS Loader.0 - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Error initializing step [OpenVPMS Loader] 2021/08/13 07:10:18 - XML output.0 - Opening output stream in encoding: UTF-8 2021/08/13 07:10:18 - ClientTransform - Step [CSV file input.0] initialized flawlessly. 2021/08/13 07:10:18 - ClientTransform - Step [XML output.0] initialized flawlessly. 2021/08/13 07:10:18 - ClientTransform - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Step [OpenVPMS Loader.0] failed to initialize! 2021/08/13 07:10:18 - Spoon - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : ClientTransform: preparing transformation execution failed 2021/08/13 07:10:18 - Spoon - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : org.pentaho.di.core.exception.KettleException: 2021/08/13 07:10:18 - Spoon - We failed to initialize at least one step. Execution can not begin! 2021/08/13 07:10:18 - Spoon - 2021/08/13 07:10:18 - Spoon - 2021/08/13 07:10:18 - Spoon - at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1286) 2021/08/13 07:10:18 - Spoon - at org.pentaho.di.ui.spoon.trans.TransGraph$30.run(TransGraph.java:4281) 2021/08/13 07:10:18 - Spoon - at java.lang.Thread.run(Thread.java:748) 2021/08/13 07:10:18 - ClientTransform - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Errors detected! 2021/08/13 07:10:18 - ClientTransform - ERROR (version 9.0.0.0-423, build 9.0.0.0-423 from 2020-01-31 04.53.04 by buildguy) : Errors detected! 2021/08/13 08:02:17 - Spoon - Spoon has ended. stopping

Re: Kettle Transforms

Submitted by tanderson on Fri, 13/08/2021 - 19:56.

1. Do you see anything different in the Logging tab?
2. Can you verify your transforms successfully? Actions -> Verify or F11
3. If these check out OK, send me your transform + a dummy input CSV to tanderson[at]openvpms[dot]org and I will try and reproduce locally.

Re: Kettle Transforms

Submitted by CWearne on Fri, 13/08/2021 - 21:33.

Thanks again for the trouble you are putting in to this.

1. The logging tab is exactly the same. The plugin is failing to initialize due to failure to load application context.

2. None of the steps verify. They all fail with the same message failure to load application context.

 

I think I am doing something really wrong with the kettle plugin. I'm not sure because I have never seen a working example but I am guessing it should provide the target value archtypes in the database so that I have something to map to and yet there is nothing. When I click 'get fields' it responds with the fields from the csv but nothing from the mysql-openvpms database. I have attached a screenshot so you know what I mean.

AttachmentSize
Screenshot from 2021-08-13 12-31-27.png 168.8 KB

Re: Kettle Transforms

Submitted by tanderson on Fri, 13/08/2021 - 22:33.

Try the minimalist example below.
The test.txt is a CSV containing 2 products.
The test.ktr has two steps; one is a CSV file input, the other an OpenVPMS Loader step.

For the CSV file input step, change the Filename field to point to where you stored test.txt.

For the OpenVPMS Loader step, edit the Connection and change the Username and password.
Verify you can connect by clicking Test.

Click the Run button. This will display the Run Options window. Click Run.

The transform should load two products, Product A and Product B.

If it fails, and doesn't provide any useful logging, try setting the Log Level to Debug in the Run Options window.

AttachmentSize
test.ktr 19.87 KB
test.txt 53 bytes

Re: Kettle Transforms

Submitted by CWearne on Fri, 13/08/2021 - 23:33.

No joy I'm afraid. I have attached the relevant screen shots and copy of the log in debug mode.

AttachmentSize
spoon.log_.txt 26.48 KB
Screenshot from 2021-08-13 14-27-52.png 142.75 KB
Screenshot from 2021-08-13 14-28-05.png 89.13 KB
Screenshot from 2021-08-13 14-29-01.png 148.64 KB

Re: Kettle Transforms

Submitted by tanderson on Sat, 14/08/2021 - 13:49.

Don't know. It should simply be a matter of:
1. extracting pdi-ce-9.0.0.0-423.zip.
   This extracts to a directory named data-integration
2. copying the mysql-connector-java-x.jar to data-integration/lib
   I use a 5.1.x version; you can use mysql-connector-java-8.x provided you use the Generic driver in Pentaho.
3. extracting openvpms-pentaho-plugin-2.2.3.zip to data-integration/plugins
4. setting JAVA_HOME to point to JDK 8.
5. running spoon.bat/spoon.sh from data-integration
    On Windows, running:

set SPOON_CONSOLE=1
.\spoon.bat
 

    forces it to run in the console window. You should see it log there as well.

If you have done all of the above and it still doesn't work, you may need to modify the plugin to add more diagnostics. The relevant class is LoaderPluginMeta

Change the getContext() method to:

/**
 * Gets the application context.
 *
 * @return the application context, or {@code null} if the load fails or
 * the database is not specified
 */
protected ApplicationContext getContext() {
    ApplicationContext context = null;
    if (database != null) {
        try {
            context = ApplicationContextMgr.getContext(database, getLog());
        } catch (Throwable exception) {
            System.err.println("Failed to load application context: " + exception.getMessage());
            exception.printStackTrace();
            logError("Failed to load application context", exception);
        }
    } else {
        System.err.println("Failed to load application context: no database");
        logError("Failed to load application context", Messages.get("LoaderPluginMeta.NoConnection"));
    }

    return context;
}

1. Rebuild openvpms-etl/openvpms-pentaho-plugin using mvn clean -DskipTests install
2. Remove the existing openvpms-pentaho-plugin directory from data-integration/plugins
3. Extract openvpms-etl/openvpms-pentaho-plugin/target/openvpms-pentaho-plugin-2.2.x.zip to data-integration/plugins
4. Run spoon.bat/.sh. If running on Windows, remember to set SPOON_CONSOLE=1

 

 

Re: Kettle Transforms

Submitted by CWearne on Sun, 15/08/2021 - 07:28.

Thanks I might try a fresh install and see if that works. Out of interest I ran pdi with java -verbose:class set and noted that PDI is using spring 4.3.* where as the openvpms plugin is on spring 5.2.*. That is not likely to be causing an issue is it?

Re: Kettle Transforms

Submitted by tanderson on Tue, 17/08/2021 - 15:16.

Shouldn't do. The plugin jars take precedence over the kettle jars.

Re: Kettle Transforms

Submitted by CWearne on Thu, 26/08/2021 - 04:47.

Ta. Just to keep you abreast of things, I did a clean install of openvpms and kettle today on a machine that's never been used for development work so no extraneous jars littering the class path etc.

The kettle transform failed in exactly the same way with same error details - failure to load application context.

I've not had a chance to rebuild the plugin yet but will make that my next step though it may take a while to get a build environment set up.

Regards

Chris

 

Re: Kettle Transforms

Submitted by CWearne on Fri, 27/08/2021 - 17:33.

Hi Tim,

I've made the modifications to LoaderPluginMeta.

Pentaho-plugin build failed. Screenshots attached. The build failed at the same point BEFORE I made the alternations as well as I had tried building from source previously and always ran in to the same duplicate class problem.

It seems I'm not to have an easy time with this plugin... Any suggestions on what I am doing wrong with the build appreciated. Maven is not one of my strong points I'm afraid. I'm guessing that I need to add some ignore dependency clauses in the pom.xml file.

 

Kind regards

Chris

AttachmentSize
Screenshot from 2021-08-27 08-26-37.png 347.83 KB
Screenshot from 2021-08-27 08-26-02.png 328.65 KB
Screenshot from 2021-08-27 08-24-56.png 346.47 KB

Re: Kettle Transforms

Submitted by tanderson on Fri, 27/08/2021 - 20:26.

What JDK and Maven versions are you running with? E.g.:

> mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\maven\apache-maven-3.6.3
Java version: 1.8.0_191, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jdk1.8.0_191\jre
Default locale: en_AU, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Are you able to build a clean checkout the OpenVPMS master branch?

> git clone git[at]bitbucket[dot]org:OpenVPMS/openvpms.git
> cd openvpms
> mvn clean -DskipTests install

Re: Kettle Transforms

Submitted by CWearne on Fri, 27/08/2021 - 23:22.

Apache Maven 3.6.0
Maven home: /usr/share/maven
Java version: 1.8.0_292, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux", version: "5.3.0-62-generic", arch: "amd64", family: "unix"

 

No - the build fails with dependency issues.

 

Re: Kettle Transforms

Submitted by tanderson on Fri, 27/08/2021 - 23:45.

OK. Which dependencies can't it download?

 

Re: Kettle Transforms

Submitted by CWearne on Tue, 31/08/2021 - 22:45.

Attached is the output from the mvn install command.

 

Edit - > I forgot to revert two changes I made to the pom.xml file before posting that log. I have since reverted the changes and rerun the build. Here is the output.

[INFO] Reactor Summary for OpenVPMS 2.2.3-SNAPSHOT:
[INFO]
[INFO] OpenVPMS ........................................... SUCCESS [  2.261 s]
[INFO] openvpms-framework-api ............................. SUCCESS [  1.721 s]
[INFO] OpenVPMS Framework ................................. SUCCESS [  5.977 s]
[INFO] OpenVPMS Maven Plugins ............................. SUCCESS [  0.010 s]
[INFO] OpenVPMS Archetype Maven Plugin .................... SUCCESS [  1.396 s]
[INFO] OpenVPMS Data Maven Plugin ......................... SUCCESS [  0.943 s]
[INFO] openvpms-db ........................................ SUCCESS [  1.297 s]
[INFO] openvpms-db-plugin ................................. SUCCESS [  0.603 s]
[INFO] OpenVPMS Archetypes ................................ SUCCESS [  6.526 s]
[INFO] OpenVPMS Reporting Framework ....................... FAILURE [ 12.149 s]
[INFO] OpenVPMS Report Maven Plugin ....................... SKIPPED
[INFO] openvpms-domain .................................... SKIPPED
[INFO] openvpms-domain-api ................................ SKIPPED
[INFO] openvpms-domain-impl ............................... SKIPPED
[INFO] openvpms-sms ....................................... SKIPPED
[INFO] openvpms-sms-api ................................... SKIPPED
[INFO] OpenVPMS plugins ................................... SKIPPED
[INFO] openvpms-plugin-manager ............................ SKIPPED
[INFO] openvpms-sms-impl .................................. SKIPPED
[INFO] openvpms-plugin-services ........................... SKIPPED
[INFO] OpenVPMS webservice util ........................... SKIPPED
[INFO] openvpms-version ................................... SKIPPED
[INFO] openvpms-mapping-api ............................... SKIPPED
[INFO] openvpms-plugin-services-impl ...................... SKIPPED
[INFO] OpenVPMS Clickatell plugin ......................... SKIPPED
[INFO] OpenVPMS ESCI adapter .............................. SKIPPED
[INFO] openvpms-test-plugins .............................. SKIPPED
[INFO] openvpms-test-service .............................. SKIPPED
[INFO] openvpms-test-plugin ............................... SKIPPED
[INFO] openvpms-test-servlet-plugin ....................... SKIPPED
[INFO] openvpms-insurance ................................. SKIPPED
[INFO] openvpms-insurance-api ............................. SKIPPED
[INFO] openvpms-test-insurance-service .................... SKIPPED
[INFO] openvpms-test-listener-plugin ...................... SKIPPED
[INFO] openvpms-test-rest-plugin .......................... SKIPPED
[INFO] openvpms-auth-plugin ............................... SKIPPED
[INFO] openvpms-plugin-framework-bundles .................. SKIPPED
[INFO] openvpms-plugin-manager-impl ....................... SKIPPED
[INFO] OpenVPMS Deputy plugin ............................. SKIPPED
[INFO] openvpms-booking ................................... SKIPPED
[INFO] openvpms-booking-service ........................... SKIPPED
[INFO] openvpms-laboratory ................................ SKIPPED
[INFO] openvpms-laboratory-api ............................ SKIPPED
[INFO] openvpms-web ....................................... SKIPPED
[INFO] OpenVPMS HL7 ....................................... SKIPPED
[INFO] openvpms-laboratory-impl ........................... SKIPPED
[INFO] openvpms-sample-laboratory-service ................. SKIPPED
[INFO] openvpms-insurance-impl ............................ SKIPPED
[INFO] openvpms-macro ..................................... SKIPPED
[INFO] openvpms-web-resource .............................. SKIPPED
[INFO] openvpms-web-echo .................................. SKIPPED
[INFO] openvpms-webdav .................................... SKIPPED
[INFO] openvpms-web-component ............................. SKIPPED
[INFO] openvpms-smartflow ................................. SKIPPED
[INFO] openvpms-web-workspaces ............................ SKIPPED
[INFO] OpenVPMS ETL ....................................... SKIPPED
[INFO] OpenVPMS ETL Load Module ........................... SKIPPED
[INFO] openvpms-web-jobs .................................. SKIPPED
[INFO] OpenVPMS Web App ................................... SKIPPED
[INFO] openvpms-pentaho-plugin ............................ SKIPPED
[INFO] openvpms-etl-extract ............................... SKIPPED
[INFO] openvpms-tools ..................................... SKIPPED
[INFO] OpenVPMS Release ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  33.675 s
[INFO] Finished at: 2021-08-31T13:41:42+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project openvpms-reports: Could not resolve dependencies for project org.openvpms:openvpms-reports:jar:2.2.3-SNAPSHOT: Failed to collect dependencies at net.sf.jasperreports:jasperreports-functions:jar:6.2.0 -> net.sf.jasperreports:jasperreports:jar:6.2.0 -> org.olap4j:olap4j:jar:0.9.7.309-JS-3: Failed to read artifact descriptor for org.olap4j:olap4j:jar:0.9.7.309-JS-3: Could not transfer artifact org.olap4j:olap4j:pom:0.9.7.309-JS-3 from/to milton-repo (http://dl.bintray.com/milton/Milton): Authorization failed for http://dl.bintray.com/milton/Milton/org/olap4j/olap4j/0.9.7.309-JS-3/ola... 403 Forbidden -> [Help 1]

 

 

AttachmentSize
mvnOutput.txt 286.17 KB

Re: Kettle Transforms

Submitted by tanderson on Tue, 31/08/2021 - 22:58.

Looks like the jaspersoft 3rd party dependencies repository has changed.

Try updating the root pom.xml from:

         <repository>
             <id>jaspersoft-third-party</id>
             <url>http://jaspersoft.artifactoryonline.com/jaspersoft/third-party-ce-artifacts/</url>
             <snapshots>
                 <enabled>false</enabled>
             </snapshots>

 to:


        <repository>
             <id>jaspersoft-third-party</id>
             <url>https://jaspersoft.jfrog.io/artifactory/third-party-ce-artifacts</url>
             <snapshots>
                 <enabled>false</enabled>
             </snapshots>

Re: Kettle Transforms

Submitted by CWearne on Thu, 02/09/2021 - 06:46.

Thanks. That all built fine. I have replaced the plugin with the version containing the increased debugging.

Re-ran your test case. Seems database is null:

"Failed to load application context: no database"

I have re-verified that the database test is successful and can go to explore tables and see the openvpms database table structure.

Just to confirm-the webapp works, I can log in see archetypes etc etc.

Is there any information on the mysql setup that I can provide you with that might help?

Kind regards

Chris

 

Re: Kettle Transforms

Submitted by tanderson on Thu, 02/09/2021 - 09:58.

The database can be null if the database name in the mapping doesn't correspond to one of those that is configured in PDI. PDI supplies the plugin with the list of available databases, and the appropriate one is looked up by name. The lookup occurs in LoaderPluginMeta:

  • If you are loading your transform from XML
public void loadXML(Node stepnode, List<DatabaseMeta> databases, IMetaStore metaStore) throws KettleXMLException {
    try {
        Node node = XMLHandler.getSubNode(stepnode, "mappings");
        mappings = (Mappings) Unmarshaller.unmarshal(Mappings.class, node);
        database = DatabaseMeta.findDatabase(databases, mappings.getConnection());  // <--- HERE
  • If you are loading your transform from a repository:
public void readRep(Repository repository, IMetaStore metaStore, ObjectId stepId, List<DatabaseMeta> databases)
        throws KettleException {
    mappings = new Mappings();
    String connection = this.repository.getStepAttributeString(stepId, CONNECTION);
    mappings.setConnection(connection);
    database = DatabaseMeta.findDatabase(databases, connection);  // <--- HERE

If your names are correct, then you'll either need to add more diagnostic logging or attach a debugger to see what's going on.

 

Re: Kettle Transforms

Submitted by CWearne on Thu, 02/09/2021 - 19:43.

Solved!! Thanks for all your time Tim.

If this happens to anyone else:

The database connection was not being stored in the xml of the ktr file so DatabaseMeta was returning null as mappings.getConnection() was returning null.

I solved the issue by following a post on sharing the database connection:

https://wiki.pentaho.com/display/EAI/Beginners+FAQ#BeginnersFAQ-Howtomak...

This resulted in the database connection being stored correctly in the xml.

 

Chris

Syndicate content