support for openssh default key format in jsch fork

As of release 7.8 of openssh the default format of generated keys is the OPENSSH PRIVATE KEY format.

 * ssh-keygen(1): write OpenSSH format private keys by default
   instead of using OpenSSL's PEM format. The OpenSSH format,
   supported in OpenSSH releases since 2014 and described in the
   PROTOCOL.key file in the source distribution, offers substantially
   better protection against offline password guessing and supports
   key comments in private keys. If necessary, it is possible to write
   old PEM-style keys by adding "-m PEM" to ssh-keygen's arguments
   when generating or updating a key.

This means, that the private key looks like this:

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABCnz7UO3z
wPva3ZeqAv3Fb3AAAAEAAAAAEAAABoAAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlz
dHAyNTYAAABBBKHuAe5N1uLPUpY3t5kyYuISOxUobPZfK8H+CQaJTCALTMFrT63UDDYLyI
2xroS67T2bWHkuhX1BHiTGP6JpwL8AAACwwZ1jHlWJTZUwle+U8cXZx2Od0s4Y71qL9onX
/+g+UVxaBeAZq88S8fbIPc1netiue9VVo7Qiw4e4WEPUCWKykQ+mwBX9798q/QUbrK3UZl
509ZLFe/DN41mVGO6rU6NHy+1mVstStgEEle7Dc+JaZwa/iZRVeRNq58MXQ5HBAO8fi4Y5
yhe050OWLkNylLtLOmRffRWo6eFg/DwCK64G5+qsSeTYVrt0nq/ffcvyovc=
-----END OPENSSH PRIVATE KEY-----

If you are a user of the java library jsch, which is a java implementation of the ssh protocol, you were not able to use this keys, because it does not support it. you either had to create your keys with a flag or convert them to old pem format.

Now I have implemented support for the “proprietary” key format in the fork of jsch, which you can find at https://github.com/mwiede/jsch

Please upgrade to version 0.1.60 and give it a try.

the future of jsch without ssh-rsa

With the release notes of openssh 8.3 it is clear, that some day in the future, servers will not accept the ssh-rsa signature algorithm any more. It will be disabled by default, because the risk is too high, that people spend money for attacks to break it. Instead, rsa-sha2-256 or rsa-sha2-512 should be used, amongst others.

At the moment, some people ask about the Jsch library, a popular java SSH implementation, because it’s furture is unclear. Unfortunately there is no answer on the sourceforge mailing list and I also tried to reach out to jcraft, it’s original author, via email but did not receive an answer yet.

Because of the popularity of Jsch, I think a lot of people will be interested in new features and additional signature algorithms being released, because without support, they cannot use the current version 0.1.55 from November 2018 any more. When servers will not accept ssh-rsa any more and no alternative host keys are set up, connections will not work any more!

Then the question is, whether Jcraft will continue to maintain it, whether some fork will take it over or that projects will have to switch to other libraries, which are maintained more actively (like sshj). But do you want to spend the time and effort to switch to a new API if you can avoid it?

Speaking about forks. When I was looking for a ssh library to support forwarding unix sockets, I did not find it. Then I decided for Jsch, because I knew the API from previous projects and just implemented it myself. But after looking at the inactivity on the sourceforge platform, I decided to create a fork of Jsch on Github. And after receiving pull requests about more supported algorithms, I realized, what was going on and what it means for the future. And that is why I am writing this post actually.

When I was searching for a fork to contribute to, I did not find a useful one for Jsch. There is https://github.com/vngx/vngx-jsch which improved javadoc amongst other things, but at least this was released to maven central 8 years back. All other projects (https://github.com/gaoxingliang/JSch or https://github.com/is/jsch), that contain the sourcecode of Jsch, did not publish the code in the form, that others can embed it into their projects. Now the question is: Did they not publish it because of legal rights or the license? As far as I understand, the BSD license allows publishing it as long as you keep the original copyright notices within the artifact.

So finally my goal of setting up a fork was to make it useful to the community. And this is by releasing it to maven central. Please check https://github.com/mwiede/jsch for the latest released version.

The benefits of having this setup are the following:

  • everybody can use it right away by setting up the artifact with maven or gradle or kotlin.
  • drop-in replacement: because the code and the artifact are forks, the package name, the inner class names and the API remain the same.
  • open for contribution, because it is hosted on GitHub. I think most of the people are used to Git nowerdays, so contribution on sourceforge is not attractive.
  • upstream compatible: in the case, that Jcraft will jump in again and will continue to provide maintenance and releases, it will be easy to give it back in their hands.

So let’s see, how fast openssh will continue with its announced plans to disable ssh-rsa in the near future.

Recommendation If you are a user of Jsch, please have a look at your servers, and if in doubt, switch your maven or gradle coordinates to the releases of the fork I created.

Maven users, replace

<dependency>
  <groupId>com.jcraft</groupId>
  <artifactId>jsch</artifactId>
  <version>0.1.55</version>
</dependency>

with

<dependency>
  <groupId>com.github.mwiede</groupId>
  <artifactId>jsch</artifactId>
  <version>0.1.58</version>
</dependency>

Gradle users, replace

implementation 'com.jcraft:jsch:0.1.55'

with

implementation 'com.github.mwiede:jsch:0.1.58'

Happy coding!

encoding in jdk openj9 docker images

Recently I was looking into encoding problems and I was really surprised to find this:

docker run –rm adoptopenjdk/openjdk11-openj9:jre-11.0.2.9_openj9-0.12.1-alpine java -XshowSettings 2>&1 | grep encoding

file.encoding = ANSI_X3.4-1968
file.encoding.pkg = sun.io
ibm.system.encoding = ANSI_X3.4-1968
os.encoding = ANSI_X3.4-1968
sun.io.unicode.encoding = UnicodeLittle
sun.jnu.encoding = ANSI_X3.4-1968

Even adding environment variables like LC_ALL or LANG did not help ūüôĀ

docker run –rm -e LANG=en_US.UTF-8 -e LC_ALL=en_US.UTF-8 adoptopenjdk/openjdk11-openj9:jre-11.0.2.9_openj9-0.12.1-alpine java -XshowSettings 2>&1 | grep encoding

But the issue was fixed with a more recent version of the image:

docker run –rm adoptopenjdk/openjdk11-openj9:jre-11.0.6_10_openj9-0.18.1-alpine java -XshowSettings 2>&1 | grep encoding

file.encoding = UTF-8
file.encoding.pkg = sun.io
ibm.system.encoding = UTF-8
os.encoding = UTF-8
sun.io.unicode.encoding = UnicodeLittle
sun.jnu.encoding = UTF-8

Be aware of this insides of the images you use!

Happy coding

fault tolerance for jax-ws with resilience4j

In a recent project, we still have to use SOAP webservices and I wanted to apply some resilience pattern such as retry to my project.
Also a collegue just presented a java library called resilience4j so I wanted to use that one.

Of course, there are a lot of other possibilites like using other libraries (like Hystrix) or applying the sidecar pattern outside of my application in a cluster.

As you can see in the documentation, resilience4j is build for functional programming style and it supports some functional interfaces
which can be decorated to apply the retry mechanism to the function invocation. In the examples, you can always find a simple setup to pass the supplier and decorate it only for the particular method.

In my use case, I do not want to write the decoration code for each and every method (or function). I have a third party WSDL, generated the webservice interface and port via wsimport and now I want to generically apply the retry mechanism to the webservice client. But my solution is not very complicated, it is using a reflection proxy and invocation handler to direcly decorate and execute the method via the Retry classes.

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Proxy;
import io.github.resilience4j.retry.Retry;
public final class WebserviceFactory{
static <T> T decorateWithRetryer(final T service, Retry retry) {
InvocationHandler invocationHandler = (proxy, method, args) -> retry.executeCheckedSupplier(() -> method.invoke(service, args));
return (T) Proxy.newProxyInstance(service.getClass().getClassLoader(),
service.getClass().getInterfaces(), invocationHandler);
}
}

This way I am able to decorate my whole service interface.

One note I can make regarding the exception handling: In case of an exception, an InvocationTargetException is thrown. If you want to have the target one, you have to unwrap it.

The source code of my example is available at https://github.com/mwiede/jaxws-resilience4j.

Connecting Spring and Hibernate though BeanContainer

Recently I was working on a Spring Boot application and I wanted to put together Spring and Hibernate with JPA EntityListeners.

As you may know, any JPA EntityListener class is not in the scope of Spring or any Ioc, as long as you bring them together. If you have the requirement to do this, there are a lot of examples in blogs, describing workarounds like

http://esus.com/hibernate-4-integrator-pattern-springs-di/ or https://guylabs.ch/2014/02/22/autowiring-pring-beans-in-hibernate-jpa-entity-listeners which in the end are doing autowiring programmatically (i.e. by using org.springframework.web.context.support.SpringBeanAutowiringSupport).

In the recent versions of Spring and Hibernate, this work became obsolete (as I described also on https://stackoverflow.com/questions/47941717/spring-dependency-injection-into-jpa-entity-listener/55452014#55452014).

Since Hibernate 5.3 there is a so called org.hibernate.resource.beans.container.spi.BeanContainer, which makes Hibernate aware of Spring or CDI beans. Later, org.springframework.orm.hibernate5.SpringBeanContainer was added with Spring 5.1, so you do not need to handle autowiring any more. See details of this feature in https://github.com/spring-projects/spring-framework/issues/20852

As an example of a JPA EntityListener, you can add the @Component annotation and wire any Spring bean to it.

@Component

public class MyEntityListener{

  private MySpringBean bean;

  @Autowired

  public MyEntityListener(MySpringBean bean){

    this.bean = bean;

  }

  @PrePersist

  public void prePersist(final Object entity) {

¬†¬†¬† …

  }

}

To make Hibernate aware of this component bean, there are two scenarios:

1) You are using Spring Boot: Then you do not have to do anything, because the configuration of LocalContainerEntityManagerFactoryBean is done automatically in org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration.

2) Outside of Spring Boot, you have to register SpringBeanContainer to Hibernate like this:

@Configuration

….

    @Bean

    public LocalContainerEntityManagerFactoryBean entityManagerFactory(ConfigurableListableBeanFactory beanFactory) {

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† LocalContainerEntityManagerFactoryBean emfb = …

            emfb.getJpaPropertyMap().put(AvailableSettings.BEAN_CONTAINER, new SpringBeanContainer(beanFactory));

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† …

            return emfb;

            }

Memory Footprints of hello-world microservices

I was curious reading about openj9 as a JVM being high performance and using a low memory footprint. I was working in a project, where the environment of software systems consisted of around 30 Java Applications, a few running on tomcat, a few on weblogic and the most running on dropwizard microservice framework. The desirable goal for every developer was to start the complete platform on his local notebook and therefor a virtalbox image was build using vagrant. As there were so many applications, each microservice itself consumed around 250MB of RAM and with the number of services growing we already hit 24GB image size of the virtualbox image. I found the blogpost https://codeburst.io/microservices-in-java-never-a7f3a2540dbb which describes the same issue and that if you those java microsevices in a cloud infrastructure, you would even have to pay even more money only because of the memory footprint.

Luckily there was somebody doing a comparsion of openj9 vs hotspot VM in his two blogpots https://royvanrijn.com/blog/2018/05/openj9-jvm-shootout/ and https://royvanrijn.com/blog/2018/05/openj9-hotsport-specjvm2008/ already. He was is showing some benchmarks and results.

My own simple comparison

I tested it on my own (just for curiosity) and came to the same result. In terms of memory consumption, there is a potential improvement with openj9 as JVM alternative.

I picked three helloworld examples using maven archetypes from dropwizard, helidon and Spring Boot.

First I packacked them into docker images using adoptopenjdk/openjdk8-openj9:alpine-slim on the one hand and adoptopenjdk/openjdk8:alpine-slim on the other hand.

Then I compared memory consumption using application metrics and dockers stats and could have a rough idea of the differences. Here is the result of the dropwizard app:

openj9 without any parameters to the jvm:

  • jvm.memory.heap.used = 10563328 B, jvm.memory.total.used = 37838048
  • docker container = 48.55MiB

hotspot without any parameters to the jvm:

  • jvm.memory.heap.used = 28511520 B, jvm.memory.total.used = 61381328
  • docker container = 118.3MiB

Conclusion

Whether you have the goal of running your complete software on your developer laptop or you want to save money running your services in a public cloud, openj9 provides a possibility to reduce the memory footprint of your java application by around 50%.

As it was mentioned in the other blog posts, there are also a few downsides with this, but if you test everything and the requirements (in terms of computation performance) are met, you should give it a try.

timezones known by java are not known by Oracle Database

Recently we had an issue with database connection, which led to the following eror:

ORA-00604: error occurred at recursive SQL level 1 ORA-01882: timezone region not found

Finally we found out, that the timezone coming from the operating system was known to Java JDK, but not to the Oracle Database we were using.

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 – 64bit Production
PL/SQL Release 12.2.0.1.0 – Production
CORE 12.2.0.1.0 Production
TNS for Linux: Version 12.2.0.1.0 – Production
NLSRTL Version 12.2.0.1.0 – Production

These are the timezones which will lead to the issue:

Africa/Juba
America/Creston
America/Kralendijk
America/Lower_Princes
America/Metlakatla
America/North_Dakota/Beulah
America/Santarem
America/Sitka
Antarctica/Rothera
Antarctica/Vostok
Asia/Hebron
Asia/Novokuznetsk
Etc/UCT
Etc/UTC
Etc/Universal
Etc/Zulu
Europe/Busingen
Pacific/Chuuk
Pacific/Pohnpei
UCT
Universal
Zulu

 

How to solve it:

  • choose a timezone id known by Oracle database
  • set environment variable i.e. export TZ=UTC or
  • set java property -Duser.timezone=UTC

 

Response validation when using Feign as http client wrapper

If you have the requirement of validating the response of a rest interface, a good opportunity in Java is to use the bean validation api and its annotations. On top, if you use OpenFeign wrapper to define the client in your application, you can now assign those validation annotations directly to your interface.

If you have already seen in my previous blogpost about Feign Outbound metrics, I am trying to extend feign with some decorator classes to add missing functionality. metrics-feign is a library, which can be used to report outbound-metrics out of any invocation to the feign wrapper.

Now I created another library, which makes it possible to validate the response using bean validation api (JSR349). Its called feign-validation and you can find it at github or maven central.

The only thing you have to do after setting it up is to add annotations to the interface class you are using. Here is an example:

interface GitHub {
@RequestLine("GET /repos/{owner}/{repo}/contributors")
@Size(min = 1, max = 3)
@Valid
List contributors(@Param("owner") String owner, @Param("repo") String repo);

}

static class Contributor {
@NotNull
@Size(min = 10)
String login;
int contributions;
}

public static void main(String... args) {

Validator validator = Validation.buildDefaultValidatorFactory().getValidator();

GitHub github = ExtendedFeign.builder(validator)//
.decoder(new GsonDecoder()).target(GitHub.class, "https://api.github.com");

// Fetch and print a list of the contributors to this library.
try {
List contributors = github.contributors("OpenFeign", "feign");
for (Contributor contributor : contributors) {
System.out.println(contributor.login + " (" + contributor.contributions + ")");
}
} catch (ConstraintViolationException ex) {
ex.getConstraintViolations().forEach(System.out::println);
}

}

If you would run this class, you can see, that the response is being validated against the given annotations and all violations a printed to stdout.

  • First, the @Size annotation leads to a constraint violation, because the size of the list is checked, whether it has at least one or maximim three items.
  • Because of @Valid, each item in the list of contributors is validated
  • @Size in Contributor class is set to verify, that the login name must have at least 10 characters. Because not all contributors have such a long name, a few more violations are reported
  • as seen, you can use any annotation from the api to fulfil the need

So what are you waiting for?

  • add
    <dependency>
      <groupId>com.github.mwiede</groupId>
      <artifactId>feign-validation</artifactId>
      <version>1.0</version>
    </dependency>

    and

    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-validator</artifactId>
        <version>5.4.2.Final</version>
    </dependency>

    to your dependencies

  • exchange¬†feign.Feign.builder¬†with¬†com.github.mwiede.feign.validation.ExtendedFeign.builder
  • and add annotations like explained above.

Deploy an app to Heroku using Wildfly and Postgresql driver

Since a Postgres Database is the most simple persistent datastore at Heroku, I was trying to setup a jee application example using this SQL database. If you have already read some of my previous blogposts, I am using Buildpack API to deploy the application to a Dyno.

Since I already got an example working with Wildfly and Mysql, it was an easy step to conduct a buildpack to install the Postgresql driver into the Wildfly container. It’s now available at¬†https://github.com/mwiede/heroku-buildpack-wildfly-postgresql.

I also provide an example of the application, which uses maven properties to extract the database url and credentials from system properties and use them in persistence.xml.

Here are the build and deploy steps to make it work (compare to https://github.com/mwiede/greeter#usage)

$ git clone https://github.com/mwiede/greeter.git
$ cd greeter
$ heroku create
$ heroku addons:create heroku-postgresql:hobby-dev
$ heroku buildpacks:clear
$ heroku buildpacks:add heroku/java
$ heroku buildpacks:add https://github.com/mwiede/heroku-buildpack-wildfly
$ heroku buildpacks:add https://github.com/mwiede/heroku-buildpack-wildfly-postgresql
$ git push heroku master

encode special characters using maven-antrun-plugin

Recently I was deploying an java web application on heroku, but one of the fields, which were picked up from system properties to be used during the maven build to set the credentials of the datasource, contained a non escaped character, which brought the deployment to fail:

21:37:17,355 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC000001: Failed to start service jboss.deployment.unit."ROOT.war".PARSE: org.jboss.msc.service.StartException in service jboss.deployment.unit."ROOT.war".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment "ROOT.war"

        at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:154)

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: Unexpected character '<' (code 60) (expected a name start character)

 at [row,col {unknown-source}]: [25,26]

        at org.jboss.as.connector.deployers.ds.processors.DsXmlDeploymentParsingProcessor.deploy(DsXmlDeploymentParsingProcessor.java:105)

        at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:147)

        ... 5 more

Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character '<' (code 60) (expected a name start character)

 at [row,col {unknown-source}]: [25,26]

        at com.ctc.wstx.sr.StreamScanner.throwUnexpectedChar(StreamScanner.java:647)

        at com.ctc.wstx.sr.StreamScanner.parseFullName(StreamScanner.java:1933)

        at com.ctc.wstx.sr.StreamScanner.parseEntityName(StreamScanner.java:2057)

        at com.ctc.wstx.sr.StreamScanner.fullyResolveEntity(StreamScanner.java:1525)

        at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2748)

        at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1073)

        at com.ctc.wstx.sr.BasicStreamReader.getElementText(BasicStreamReader.java:670)

        at org.jboss.jca.common.metadata.common.AbstractParser.rawElementText(AbstractParser.java:166)

        at org.jboss.jca.common.metadata.common.AbstractParser.elementAsString(AbstractParser.java:153)

        at org.jboss.jca.common.metadata.ds.DsParser.parseDataSource(DsParser.java:1149)

        at org.jboss.jca.common.metadata.ds.DsParser.parseDataSources(DsParser.java:177)

        at org.jboss.jca.common.metadata.ds.DsParser.parse(DsParser.java:120)

        at org.jboss.jca.common.metadata.ds.DsParser.parse(DsParser.java:79)

        at org.jboss.as.connector.deployers.ds.processors.DsXmlDeploymentParsingProcessor.deploy(DsXmlDeploymentParsingProcessor.java:90)

        ... 6 more

To come around this problem, we need to encode those characters as proper xml entities. So here is how I did this:

First, introduce the system property as maven property inside of the property section:

 <properties>
<db.url>${JDBC_DATABASE_URL}</db.url>
<db.user>${JDBC_DATABASE_USERNAME}</db.user>
<db.password>${JDBC_DATABASE_PASSWORD}</db.password>
</properties>

Second, define the maven-antrun-plugin as plugin to be executed during validate phase, the phase, before even resources are copied or filtered by maven. Important is that the flag exportAntProperties is set to true. I was not able to override an existing property, therefore I created new ones.

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>validate</phase>
<configuration>
<exportAntProperties>true</exportAntProperties>
<target>
<script language="javascript">
<![CDATA[ 

if (!String.prototype.encodeHTML) {
String.prototype.encodeHTML = function () {
return this.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&apos;');
};
}

project.setProperty("enc.db.url",project.getProperty("db.url").encodeHTML());
project.setProperty("enc.db.user",project.getProperty("db.user").encodeHTML());
project.setProperty("enc.db.password",project.getProperty("db.password").encodeHTML());

]]>
</script>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>

Third, use the properties in any resouce file while filtering:

<datasources xmlns="http://www.jboss.org/ironjacamar/schema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.jboss.org/ironjacamar/schema http://docs.jboss.org/ironjacamar/schema/datasources_1_0.xsd">
<!-- The datasource is bound into JNDI at this location. We reference
this in META-INF/persistence.xml -->
<datasource jndi-name="java:jboss/datasources/GreeterQuickstartDS"
pool-name="greeter-quickstart" enabled="true" use-java-context="true">
<connection-url>${enc.db.url}</connection-url>
<driver>postgresql</driver> 
<security>
<user-name>${enc.db.user}</user-name>
<password>${enc.db.password}</password>
</security>

</datasource>
</datasources>

See the complete code example at my github repo https://github.com/mwiede/greeter.