A Pattern for Wicket Data Providers

Apache Wicket provides the AjaxFallbackDefaultDataTable¹ for displaying a paged view of a large data set, and this component requires an implementation of ISortableDataProvider to populate the rows. This data provider has several responsibilities:

• Keep track of the current sort order.
• Store any search parameters in a Serializable form.
• Fetch a count of rows that match the search parameters
• Provide an iterator over a subset of rows that match the search parameter, ordered by the current sort.

Low Latency Video over the Internet

Flash is dying, and I’m one of the few people who’ll miss it. While Flash was and is a terrible security risk, it solved one problem almost completely by accident: sending video to many people with a low delay. Working with online auctions, delay means a lot to me. If the viewers are actually bidding in a real-time auction, a delay of a few seconds is a problem. If they’re bidding against people who are actually at the auction, it’s impossible. By the time the remote buyer hears a bid from the auction hall, buyers who are physically there can bid again several times.

The RTMP protocol used by Flash requires a direct TCP connection from the browser to the video server: this allows an end to end delay of less than a second, while making it impossible to set up a shared cache infrastructure or CDN. Modern streaming protocols such as HLS and DASH are CDN friendly, by breaking the video into segments of around 1 to 10 seconds. This segmentation forces an end-to-end delay of at least one segment duration.

It’s actually a little worse than that, because each segment should be an independent video file, and not depend on any previous file. So as the file duration gets shorter, the efficiency gets less. At the limit, if each segment was a single frame of video, it would be equivalent to sending a sequence of JPEG files.

The modern alternative is WebRTC, which provides browser APIs to set up an RTP session. Unfortunately support is currently limited to Firefox, Chrome, and Opera. Which means that the near future looks like a mixed world, with RTMP streaming to the Flash plugin in IE and Edge with WebRTC to Chrome and Firefox.

And iOS? WebRTC is marked as “in development” for WebKit, but we don’t know when, or whether this will be available on iOS Safari. Until then Apple doesn’t support any low-latency protocol in their web browser, so there’s no alternative to an app for that.

Originally published by Adrian Cox at https://adrianathumboldt.github.io/.

Laying down the law with the Maven Enforcer Plugin

We recently moved an application to a new server, with an updated installation of Java 8. It didn’t take long to discover that Amazon S3 accesses were failing with an authentication error, and it didn’t take much longer to discover that this problem was already fixed in the version of the AWS Java SDK we were using.

com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header

The real problem was that we had many dependencies on different versions of Joda Time, and AWS had lost the battle - we were actually running Joda Time 2.0.

It was time to stop this happening again, with the Maven Enforcer Plugin. It’s possible to fail the build if we ever reference two versions of the same dependency:

Spring 4 and Quartz 2 Integration with Custom Annotations

I’m integrating Quartz scheduling into an application, and was looking for an annotation based approach to configuration. I quickly found the SivaLabs implementation, which works for Quartz 1.8. I’ve made a few changes to use this with Spring 4.1.4 and Quartz 2.2.3.

The custom annotation remains the same:

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Component
@Scope("prototype")
public @interface QuartzJob {
    String name();
    String group() default "DEFAULT_GROUP";
    String cronExp();
}

The major change is that I have replaced the ApplicationListener class with a bean that performs a single scan of a selected package. I chose this approach because my configuration does not change once loaded, and I wanted to throw a fatal exception on any error in Quartz configuration. I’ve refactored to add each job to the scheduler as it is discovered, rather than building an intermediate list.

public class QuartzJobScanner
{
    @Autowired
    private Scheduler scheduler;

    private static final Logger log = LoggerFactory.getLogger(QuartzJobScanner.class);

    private final String scanPackage;

    public QuartzJobScanner(String scanPackage) {
         this.scanPackage = scanPackage;
    }

    @PostConstruct
    public void scheduleJobs() throws Exception
    {
        ClassPathScanningCandidateComponentProvider provider = new ClassPathScanningCandidateComponentProvider(false);
        // Filter to include only classes that have a particular annotation.

        provider.addIncludeFilter(new AnnotationTypeFilter(QuartzJob.class));
        // Find classes in the given package (or subpackages)

        Set<BeanDefinition> beans = provider.findCandidateComponents(scanPackage);

        for (BeanDefinition bd: beans)
            scheduleJob(bd);
    }

The CronTriggerBean and JobDetailBean from Spring 3 are gone in Spring 4. The new code builds the job using the Quartz 2 fluent builders, rather than Spring’s factory beans:

    private void scheduleJob(BeanDefinition bd) throws Exception
    {
        Class<?> beanClass = Class.forName(bd.getBeanClassName());
        QuartzJob quartzJob = beanClass.getAnnotation(QuartzJob.class);

        // Sanity check

        if(Job.class.isAssignableFrom(beanClass) && quartzJob != null)
        {
            @SuppressWarnings("unchecked") Class<? extends Job> jobClass = (Class<? extends Job>)(beanClass);

            log.info("Scheduling quartz job: " + quartzJob.name());
            JobDetail job = JobBuilder.newJob(jobClass)
                    .withIdentity(quartzJob.name(), quartzJob.group())
                    .build();
            Trigger trigger = TriggerBuilder.newTrigger()
                    .withSchedule(CronScheduleBuilder.cronSchedule(quartzJob.cronExp()))
                    .withIdentity(quartzJob.name() + "_trigger", quartzJob.group())
                    .forJob(job)
                    .build();

            scheduler.scheduleJob(job, trigger);
        }
    }
}

The job factory is unchanged from the original article, and the XML configuration looks like this:

<context:component-scan base-package="uk.co.humboldt.Application.Services" />
<bean class="uk.co.humboldt.Application.Services.QuartzJobScanner">
        <constructor-arg value="uk.co.humboldt.Application.Services"/>
</bean>
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
        <property name="jobFactory">
            <bean class="uk.co.humboldt.Application.Services.QuartzJobFactory"/>
        </property>
</bean>

At this point jobs are automatically configured as long they:

• implement the Job interface
• have the QuartzJob annotation
• are in a package scanned by Spring, and by the QuartzJobScanner

Full code here.

Originally published by Adrian Cox at https://adrianathumboldt.github.io/.

A Breaking Change

We have a monolith. It’s not the biggest monolith, but it does contain a lot of Java libraries, managed with Maven. This week an upgrade to one package caused a distant side effect.

Our monolith is a web application, and uses an embedded Jetty server. It also uses Jasper reports, compiling jrxml files on demand. This week, our builds stopped compiling reports, with an exception that Jasper could not find javac. The first thought was that somebody had changed the server install from a JDK to a JRE, leaving the compiler unavailable. But after installing the latest JDK, and checking the system path, the error changed:

net.sf.jasperreports.engine.JRException: Errors were encountered when compiling report expressions class file:
C:\Users\adrian\Workspaces\Customer\App\App-Web\j491845presale_seller_1464295746680_612367.java:227: error: cannot find symbol
                value = DATEFORMAT(((java.sql.Timestamp)field_sale_start.getValue()), "dd/MM/YYYY"); //$JR_EXPR_ID=2$
                        ^
  symbol:   method DATEFORMAT(Timestamp,String)
  location: class j491845presale_seller_1464295746680_612367

As jasperreports-functions-5.6.1.jar was in the classpath, there was clearly another problem. Why was Jasper searching for javac at all, when Eclipse JDT was available?

 <dependency>
            <groupId>org.eclipse.jdt.core.compiler</groupId>
            <artifactId>ecj</artifactId>
            <version>4.4</version>
</dependency>

The next step was to go through our commits to find the point where things went wrong. It didn’t take long to discover the breakage was caused by an upgrade from Jetty 9.0 (2013) to Jetty 9.3 (2016). But how had this broken Jasper?

Stepping through Jasper initialisation reveals that Jasper attempts to load org.eclipse.jdt.internal.compiler.env.AccessRestriction in order to discover which version of JDT is present. This attempt fails, even though the class is present on the classpath.

This turned my debugging attention to the classloader. Jetty includes a WebAppClassloader, and setting a breakpoint in its loadClass method revealed that the JDT class was identified as a server class. Knowing this cause, it didn’t take long to find the commit that broke Jasper on Github.

As our application uses Jetty as an embedded web server, I disabled the WebAppClassloader like this, and Jasper reports worked again:

WebAppContext wac = new WebAppContext();
// As we use Jasper Reports dynamic compiling, ensure Jetty uses the standard system classloader

// so that Jasper Reports can load ECJ

wac.setClassLoader(ClassLoader.getSystemClassLoader());

Originally published by Adrian Cox at https://adrianathumboldt.github.io/.