WTP 3.8.2 Released!

March 28, 2017 03:01 PM

The Web Tools Platform's 3.8.2 Release is now available! Installation and update can be performed using the Neon Update Site at http://download.eclipse.org/releases/neon/, or the Eclipse Marketplace. Release 3.8.2 fixes issues that occur in prior releases or have been reported since 3.8.1's release. WTP 3.8.2 is featured in the Neon.3 Eclipse IDE for Java EE Developers, with selected features also included in other packages. Adopters can download the build itself directly.

More news


March 28, 2017 03:01 PM

Work on IoT that Matters

by Ian Skerrett at March 28, 2017 02:45 PM

Tim O’Reilly use to talk a lot about encouraging people to ‘Work on stuff that matters‘. Unfortunately, the IoT industry is often not the best example of this principle. The twitter feed ‘Internet of Shit‘ chronicles IoT solutions that should never have been built. For IoT to be successful we need more example of how IoT can make substantial and meaningful change in our lives.

For this reason, I was encouraged and impressed by the recent winners of the Eclipse IoT Developer Challenge.  All three winners and many of the participants are working on use cases that have significant impact. For example,

Sleep analysis for mood disorders

InMoodforLife is an application to analyze sleep patterns of individuals affected by bipolar disorder. They are using off-the-shelf hardware components, open source software and deep learning research to analyze sleep patterns and then help predict mood disorders. The intent is to provide a solution that helps improve the approach for therapy and provide quicker treatment.  This solution will have a significant impact for the individuals and families.

Smarter farming

Krishi IoT is working to make farming more efficient and smarter. They are solving day-to-day issues experienced by farmers, like irrigation, crop diseases, pest protection and are building a solution that is accessible and affordable by all farmers.

Reduce your home energy bill

RHDS has taken on the challenge of creating a solution that will reduce home energy consumption, increase comfort and create a healthier living environment. They were able to demonstrate in near real-time, a solution to calculate and present the Key Metrics of a professional Energy Audit. They used algorithms from Building Science, open source software and low cost sensors to create a $200 solution that would normally be done by experts at a cost of $300-$500.

IoT Solutions that Matter

All three of these solutions are examples of Work on IoT that Matters. Congratulations to all these winners that are demonstrating the creativity and innovation that is possible through open hardware and open software. We are proud to have Eclipse IoT involved in IoT Solutions that Matter.

 



by Ian Skerrett at March 28, 2017 02:45 PM

Technical Debt: How Do You Unfork a Fork?

by Tracy M at March 28, 2017 01:06 PM

filled_cirle_point_style_graph

Everyone knows how to fork that interesting open source project, it’s simple and handy to do. What’s not so easy to do is to merge back a fork that has over the years taken on a life of its own and for many reasons has diverged drastically from the original repo.

This is a case study of an ongoing project we are doing with SWT XYGraph, a visualisation project that is now part of Eclipse Nebula. It is the story of a fork of SWT XYGraph maintained by Diamond Light Source, the UK’s national synchrotron. But mostly it is a story about the efforts to merge the fork, reduce technical debt, and work towards the goal of sharing software components for Science, a key goal of the Eclipse Science Working Group.

Know Your History

One of the first things in this project was to understand the history – spanning 8 years – of the fork.  We knew the Diamond fork was done before SWT XYGraph became part of Nebula and under the Eclipse Foundation umbrella. The fork was made in order to quickly add in a number of new features that required some fundamental architectural changes to the code base.

However on looking through the history, we found there were more than just 2 forks involved. The original project had been developed as part of Control System Studio (CSS) from Oakridge National Labs. CSS had in turn been forked by Diamond and customised for the local facility. Even though SWT XYGraph had been contributed to the Eclipse Nebula project, the original repo and many, many forks were still out there: more than enough forks for a dinner party. I can’t explain it any further in words so will dump our illegible working diagram of it all here:

forks

Patches were pulled across and merged across forks when it was straightforward to do so. But with so many forks, this was a case where git history really mattered. Anywhere the history was preserved it was straightforward to track the origins of a specific feature – much harder in the cases where the history was lost. Git history is important, always worth some effort to preserve.

Choose Your Approach Carefully

Deciding if it worthwhile to merge a big fork takes some consideration. The biggest question to ask is: Are the architectural changes fundamentally resolvable? (Not like Chromium’s fork of Webkit – Blink). If that is a yes, then it’s a case of trading off the long-term benefits for the short term pain. In this case, Diamond knew it was something they wanted to do, more a matter of timing and picking the correct approach.

Together there seemed to be 2 main ways to tackle removing the fork that was part of a mature product in constant use at the scientific facility.

Option 1: Create a branch and work in parallel to get the branch working with upstream version, then merge the branch.

Option 2: Avoid a branch, but work to incrementally make the fork and upstream SWT XYGraph plug-ins identical, then make the switch over to the upstream version.

Option 1 had been tried before without success; there were too many moving parts and it created too much overhead, and ironically another fork to maintain. So it was clear this time Option 2 would be the way forward.

Tools are Your Friend

The incremental merging of the two needed to be done in a deliberate, reproducible manner to make it easier to trace back any issues coming up. Here are the tools that were useful in doing this.

1. Git Diff

The first step was to get an idea of the scale of the divergence, both quantitatively and qualitatively.

For quantity, a rough and ready measure was obtained by using git diff:

$ git diff --shorstat <diamond> <nebula>
399 files changed, 15648 insertions(+), 15368 deletions(-)

$ git diff <diamond> <nebula> | wc -l
37874

2. Eclipse IDE’s JDT formatter

Next, we needed to remove diffs that were just down to formatting. For this using Eclipse IDE and the quick & easy formatting. Select “src” folder, choose Source menu -> Format. All code formatted to Eclipse standard in one go.

format_src_folder

3. Merge Tools

Then it was time to dive into the differences and group them into features, separating quick fixes from changes that broke APIs. For this we used the free and open meld on Linux.

3. EGit Goodness

Let’s say we found a line of code different in the fork. To work out where the feature had come from, we could use ‘git blame‘ but much nicer is the eGit support in Eclipse IDE. Show annotations was regularly used to try to work out where that feature had come from, which fork it had been originally created on and then see if we could find any extra information such as bugzilla or JIRA tickets describing the feature. We were always grateful for code with good and helpful commit messages.

egit_annotations.png

3. Bug Tracking Tools

In this case we were using two different bug trackers: Bugzilla on the Eclipse Nebula side of things and JIRA on the Diamond side of things. As part of the merge, we were contributing lots and lots of distinct features to Nebula, we had a parent issue: Bug 513865 to which we linked all the underlying fixes and features, aiming to keep each one distinct and standalone. At the time of writing that meant 21 dependent bugs.

4. Gerrits & Pull Requests

Gerrits were created for each bug for Eclipse Nebula. Pull requests were created for each change going to Diamond’s DAWN (over 50 to date). Each was reviewed before being committed back. In many cases we took the opportunity to tidy code up or enhance it with things like standalone examples that could be used to demonstrate the feature.

5. Github Built-in Graphs

It was also good to use the built in Github built in Graphs  (on any repository click on ‘Graphs’ tab), first to see other forks out in the wild (Members tab):

members

Then the ‘Network’ tab to keep track of the relationship with those forks compared to the main Diamond fork:

networkgraph

Much nicer than our hand-drawn effort from earlier, though in this case not all the code being dealt with was in Github.

Win/Win

The work is ongoing and we are getting to the tricky parts – the key reasons the forks were created in the first place – to make fundamental changes to the architecture. This will require some conversations to understand the best way forward. Already with the work that has been done, there has been mutual benefits: Diamond get new features and bug fixes developed in the open source and Eclipse Nebula get new features and bug fixes developed at Diamond Light Source. The New & Noteworthy for Eclipse Nebula shows off screenshots of all the new features as a result of this merge.

Nebula_N&N_1.3_-_improved_mouse_cursors

Going forward this paves the way for Diamond to not only get rid of duplicate maintenance of >30,000 lines of Java code (according to cloc), but to contribute some significant features they have developed that integrate with SWT XYGraph. In doing so with the Eclipse Science Working Group it make a great environment to collaborate in open source and make advancements that benefit all involved.



by Tracy M at March 28, 2017 01:06 PM

Mylyn Wikitext is available on maven central

March 25, 2017 11:00 PM

As announced in my previous blog post, important refactorings are going in the mylyn wikitext project. One of the first goals is achieved: the jars are now available on maven central (see also Mylyn WikiText on Maven Central by David Green). Have a look at the org.eclipse.mylyn.docs group in a maven central viewer (right now the newest version is 3.0.6).

Wikitext provides the capability to transform from one language to another. I have already blogged about the possibility to convert to AsciiDoc from Eclipse IDE (new feature introduced with Oxygen M4).

Convert from textile to AsciiDoc with Eclipse IDE

Let demonstrates how you can convert from MediaWiki to AsciiDoc, directly in a small Java program. It is just a few lines of code in a simple main class.

From MediaWiki to Asciidoctor (logos)

In the Example the content of the Scout/Contribution page taken from the Eclipse wiki will be converted into an AsciiDoc File: scout_contribution.adoc. The code is straight forward:

Listing 1. Example main class
import java.io.File;
import java.io.StringWriter;
import java.net.URL;

import org.eclipse.mylyn.wikitext.asciidoc.internal.AsciiDocDocumentBuilder;
import org.eclipse.mylyn.wikitext.mediawiki.MediaWikiLanguage;
import org.eclipse.mylyn.wikitext.parser.DocumentBuilder;
import org.eclipse.mylyn.wikitext.parser.MarkupParser;

import com.google.common.base.Charsets;
import com.google.common.io.Files;
import com.google.common.io.Resources;

public class MediaWikiToAdocExample {

  public static void main(String[] args) throws Exception {
    String url = "http://wiki.eclipse.org/index.php?title=Scout/Contribution&action=raw&templates=expand";
    String input = Resources.toString(new URL(url), Charsets.UTF_8);

    MediaWikiLanguage markupLanguage = new MediaWikiLanguage();
    markupLanguage.setInternalLinkPattern("https://wiki.eclipse.org/index.php?title={0}");
    MarkupParser parser = new MarkupParser(markupLanguage);

    StringWriter writer = new StringWriter();
    DocumentBuilder builder = new AsciiDocDocumentBuilder(writer);
    parser.setBuilder(builder);
    parser.parse(input);

    String output = "= Eclipse Scout - Contribution\n\n";
    output += writer.toString();
    Files.write(output, new File("scout_contribution.adoc"), Charsets.UTF_8);
  }
}

On your classpath, you will need some of the wikitext jars and google guava. The pom.xml File for Maven looks like this:

Listing 2. pom.xml for the Example
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>wikitext.examples</groupId>
    <artifactId>mediawiki-to-adoc</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.eclipse.mylyn.docs</groupId>
            <artifactId>org.eclipse.mylyn.wikitext</artifactId>
            <version>3.0.6</version>
        </dependency>
        <dependency>
            <groupId>org.eclipse.mylyn.docs</groupId>
            <artifactId>org.eclipse.mylyn.wikitext.mediawiki</artifactId>
            <version>3.0.6</version>
        </dependency>
        <dependency>
            <groupId>org.eclipse.mylyn.docs</groupId>
            <artifactId>org.eclipse.mylyn.wikitext.asciidoc</artifactId>
            <version>3.0.6</version>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>21.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Here is a side by side comparison of the wiki page and of a preview of scout_contribution.adoc:

2017 03 26 mediawiki and adoc preview 800

If you look very closely at this example, you might notice some issues during the conversion process. I have reported them in Bug 508262.

If you are interessted by the Mylyn Wikitext project, you can follow @WikiText on twitter. You can ask questions on StackOverflow using the mylyn-wikitext tag. Are you looking for more usage examples of Mylyn Wikitext? Have a look at greensopinion/wikitext-examples on GitHub. David Green has prepared additional use cases there.


March 25, 2017 11:00 PM

A new “Kafka” novel : the OpenShift & Kubernetes deployment

by ppatierno at March 25, 2017 02:26 PM

This blog post doesn’t want to be an exhaustive tutorial to describe the way to go for having Apache Kafka deployed in an OpenShift or Kubernetes cluster but just the story of my journey for having a “working” deployment and using it as a starting point to improve over time with a daily basis work in progress. This journey started using Apache Kafka 0.8.0, went through 0.9.0, finally reaching the nowadays 0.10.1.0 version.

From “stateless” to “stateful”

One of the main reasons to use a platform like OpenShift/Kubernetes (let me to use OS/K8S from now) is the scalability feature we can have for our deployed applications. With “stateless” applications there are not so much problems to use such a platform for a Cloud deployment; every time an application instance crashes or needs to be restarted (and/or relocated to a different node), just spin up a new instance without any relationship with the previous one and your deployment will continue to work properly as before. There is no need for the new instance to have information or state related to the previous one.

It’s also true that, out there, we have a lot of different applications which need to persist state information if something goes wrong in the Cloud and they need to be restarted. Such applications are “stateful” by nature and their “story” is important so that just spinning up a new instance isn’t enough.

The main challenges we have with OS/K8S platform are :

  • pods are scaled out and scaled in through Replica Sets (or using Deployment object)
  • pods will be assigned an arbitrary name at runtime
  • pods may be restarted and relocated (on a different node) at any point in time
  • pods may never be referenced directly by the name or IP address
  • a service selects a set of pods that match specific criterion and exposes them through a well-defined endpoint

All the above considerations aren’t a problem for “stateless” applications but they are for “stateful” ones.

The difference between them is also know as “Pets vs Cattle” meme, where “stateless” applications are just a herd of cattle and when one of them die, you can just replace it with a new one having same characteristics but not exactly the same (of course !); the “stateful” applications are like pets, you have to take care of them and you can’t just replace a pet if it’s die 😦

Just as reference you can read about the history of “Pets vs Cattle” in this article.

Apache Kafka is one of these type of applications … it’s a pet … which needs to be handle with care. Today, we know that OS/K8S offers Stateful Sets (previously known as Pet Sets … for clear reasons!) that can be used in this scenario but I started this journey when they didn’t exist (or not released yet), so I’d like to share with you my story, the main problems I encountered and how I solved them (you’ll see that I have “emulated” something that Stateful Sets offer today out of box).

Let’s start with a simple architecture

Let’s start in a very simple way using a Replica Set (only one replica) for Zookeeper server and the related service and a Replica Set (with three replicas) for Kafka servers and the related service.

reference_architecture_1st_ver

The Kafka Replica Set has three replicas for “quorum” and leader election (even for topic replication). The Kafka service is needed to expose Kafka servers access even to clients. Each Kafka server may need :

  • unique ID (for Zookeeper)
  • advertised host/port (for clients)
  • logs directory (for storing topic partitions)
  • Zookeeper info (for connection)

The first approach is to use the broker id dynamic generation so that when a Kafka server starts and needs to connect to Zookeeper, a new broker id is generated and assigned to it. The advertised host/port are just container IP and the fixed 9092 port while the logs directory is predefined (by configuration file). Finally, the Zookeeper connection info are provided through the related Zookeeper service using the related environment variables that OS/K8S creates for us (ZOOKEEPER_SERVICE_HOST and ZOOKEEPER_SERVICE_PORT).

Let’s consider the following use case with a topic (1 partition and 3 replicas). The initial situation is having Kafka servers with broker id 1001, 1002, 1003 and the topic with current state :

  • leader : 1001
  • replicas : 1001, 1002, 1003
  • ISR : 1001, 1002, 1003

It means that clients need to connect to 1001 for sending/receiving messages for the topic and that 1002 and 1003 are followers for having this topic replicated handling failures.

Now, imagine that the Kafka server 1003 crashes and a new instance is just started. The topic description becomes :

  • leader : 1001
  • replicas : 1001, 1002, 1003 <– it’s still here !
  • ISR  : 1001, 1002 <– that’s right, 1003 is not “in-sync”

Zookeeper still sees the broker 1003 as a host for one of the topic replicas but not “in-sync” with the others. Meantime, the new started Kafka server has a new auto generated id 1004. A manual script execution (through the kafka-preferred-replica-election.sh) is needed in order to :

  • adding 1004 to the replicas
  • removing 1003 from replicas
  • new leader election for replicas

use_case_autogenerated_id.png

So what does it mean ?

First of all, the new Kafka server instance needs to have the same id of the previous one and, of course, the same data so the partition replica of the topic. For this purpose, a persistent volume can be the solution used, through a claim, by the Replica Set for storing the logs directory for all the Kafka servers (i.e. /kafka-logs-<broker-id>). It’s important to know that, by Kafka design, a logs directory has a “lock” file locked by the server owner.

For searching for the “next” broker id to use, avoiding the auto-generation and getting the same data (logs directory) as the previous one, a script (in my case a Python one) can be used on container startup before launching the related Kafka server.

In particular, the script :

  • searches for a free “lock” file in order to reuse the broker id for the new Kafka server instance …
  • … otherwise a new broker id is used and a new logs directory is created

Using this approach, we obtain the following result for the previous use case :

  • the new started Kafka server instance acquires the broker id 1003 (as the previous one)
  • it’s just automatically part of the replicas and ISR

use_case_locked_id

But … what on Zookeeper side ?

In this deployment, the Zookeeper Replica Set has only one replica and the service is needed to allow connections from the Kafka servers. What happens if the Zookeeper crashes (application or node fails) ? The OS/K8S platform just restarts a new instance (not necessary on the same node) but what I see is that the currently running Kafka servers can’t connect to the new Zookeeper instance even if it holds the same IP address (through the service usage). The Zookeeper server closes the connections after an initial handshake, probably related to some Kafka servers information that Zookeeper stores locally. Restarting a new instance, this information are lost !

Even in this case, using a persistent volume for the Zookeeper Replica Set is a solution. It’s used for storing the data directory that will be the same for each instance restarted; the new instance just finds Kafka servers information in the volume and grants connections to them.

reference_architecture_1st_ver_zookeeper

When the Stateful Sets were born !

At some point (from the 1.5 Kubernetes release), the OS/K8S platform started to offer the Pet Sets then renamed in Stateful Sets like a sort of Replica Sets but for “stateful” application but … what they offer ?

First of all, each “pet” has a stable hostname that is always resolved by DNS. Each “pet” is being assigned a name with an ordinal index number (i.e. kafka-0, kafka-1, …) and finally a stable storage is linked to that hostname/ordinal index number.

It means that every time a “pet” crashes and it’s restarted, the new one will be the same : same hostname, same name with ordinal index number and same attached storage. The previous running situation is fully recovered and the new instance is exactly the same as the previous one. You could see them as something that I tried to emulate with my scripts on container startup.

So today, my current Kafka servers deployment has :

  • a Stateful set with three replicas for Kafka servers
  • an “headless” service (so without an assigned cluster IP) that is needed for having Stateful set working (so for DNS hostname resolution)
  • a “regular” service for providing access to the Kafka servers from clients
  • one persistent volume for each Kafka server with a claim template defined in the Stateful set declaration

reference_architecture_statefulsets

Other then to use a better implementation 🙂 … the current solution doesn’t use a single persistent volume for all the Kafka servers (having a logs directory for each of them) but it’s preferred to use a persistent storage dedicated to only one “pet”.

It’s great to read about it but … I want to try … I want to play !

You’re right, I told you my journey that isn’t finished yet but you would like to try … to play with some stuff for having Apache Kafka deployed on OS/K8S.

I called this project Barnabas like one of the main characters of the author Franz Kafka who was a … messenger in “The Castel” novel :-). It’s part of the bigger EnMasse project which provides a scalable messaging as a service (MaaS) infrastructure running on OS/K8S.

The repo provides different deployment types : from the “handmade” solution (based on bash and Python scripts) to the current Stateful Sets solution that I’ll improve in the coming weeks.

The great thing about that (in the context of the overall EnMasse project) is that today I’m able to use standard protocols like AMQP and MQTT to communicate with an Apache Kafka cluster (using an AMQP bridge and an MQTT gateway) for all the use cases where using Kafka makes sense against traditional messaging brokers … that from their side have to tell about a lot of stories and different scenarios 😉

Do you want to know more about that ? The Red Hat Summit 2017 (Boston, May 2-4) could be a good place, where me and Christian Posta (Principal Architect, Red Hat) will have the session “Red Hat JBoss A-MQ and Apache Kafka : which to use ?” … so what are you waiting for ? See you there !



by ppatierno at March 25, 2017 02:26 PM

EMF Forms and EMF Client Platform 1.12.0 released!

by Maximilian Koegel and Jonas Helming at March 24, 2017 03:04 PM

We are happy to announce that together with Neon.3, we have released  EMF Forms and EMF Client Platform 1.12.0!

We want to thank our continuously active team of 12 contributors (36 contributors over all) for woking on 25 bug reports and feature requests. As the list reveals, and due to the short development cycle between Neon.2 and Neon.3, the 1.12.0 release was mainly focussed on bug fixes. The subsequent release will introduce a number of new features again, which are already in the pipeline.

We will give a talk at Eclipse Converge 2017 also introducing the new features of the latest releases, so make sure you register soon.

Please note, that we have began work on EMF Forms / ECP 2.0.0 in parallel to the 1.x development stream. We plan a final 1.x release along with Oxygen (1.13.0). Afterwards, we plan to focus on the 2.0.0 release stream. However, users do not have to worry too much about API breaks. There are two major changes that we wish to apply with 2.0.0. First, we plan to remove API, which is already marked as deprecated. So, if you still use any deprecated API, now is a good time to start refactoring here. Second, we will refactor the way “domain model references” are stored in the model. This will mainly allow us to bind to new data models. For this change, we plan to provide a migration for existing view models, so this change should be seamless for users of the framework.

As always, we will also blog about new features of the EMF Forms / ECP 1.12.0 release in the upcoming weeks! Please follow this blog or follow us on twitter to get notified about the new posts.

In case you don’t know EMF Forms: It is a framework focused on the creation of form-based UIs. EMF Client Platform is designed to support the development of applications based on an EMF data model. If you are not yet familiar with EMF Forms, please refer to this tutorial for a introduction.

Both of these frameworks are part of Eclipse Modeling Tools Neon.3, but you can also find the new release on our download pages:


TwitterGoogle+LinkedInFacebook

Leave a Comment. Tagged with eclipse, emf, emfcp, emfforms, eclipse, emf, emfcp, emfforms


by Maximilian Koegel and Jonas Helming at March 24, 2017 03:04 PM

Papyrus-RT 0.9 is available!

by tevirselrahc at March 24, 2017 01:47 PM

As I mentioned in a previous post, Papyrus for Real Time v0.9 is now available for download!

And yes, there is a Beta associated with it…but my minions tell me it is still being setup. I’ve been told something will be announced about this next week, and if you are interested in participating, you can just let me know either in comments on this blog or through Twitter @PapyrusUML. I’ll then let my minions know!

Are you curious about this release? Then have a look at the Release Notes.


Filed under: News, Papyrus-RT Tagged: release

by tevirselrahc at March 24, 2017 01:47 PM

e(fx)clipse runtime library – Working with the (JavaFX) UI-Thread

by Tom Schindl at March 23, 2017 01:08 PM

When developing UI-Application a very frequent task is

  • to synchronizes yourself back from an worker-thread to the UI-Thread
  • schedule task to run in the future
  • Block the program flow and wait for a condition being met (in SWT called spinning the event loop)

JavaFX 8 has a public API for most of those building blocks (only exception is Event-Loop-Spinning who was an internal API in Java8 and is public API in 9) but using higher level API reduces the boilerplate code you need to implement.

Before we start let’s see how you can access to it in your maven-driven projects (I’m not gradle safey enough to show how it works there) you need to

  • add the efxclipse maven repository (at the time of this writing you need to nightly repository because you need at least 3.0.0)
  • add the following dependency
    <dependency>
      <groupId>at.bestsolution.efxclipse.rt</groupId>
      <artifactId>org.eclipse.fx.ui.controls</artifactId>
      <version>3.0.0-SNAPSHOT</version>
    </dependency>
    

Let’s take a look at some of those APIs and how they can help you write better code:

Writing back from worker thread

Let’s suppose we have a service who returns a search result as a CompletableFuture and we want to push the result – once available – to JavaFX ObjectProperty:

import org.eclipse.fx.core.ServiceUtils;

// Lookup the domain service in the service registry
PersonSearchService service = 
  ServiceUtils.getService(PersonSearchService.class);

ObjectProperty<Person> person = ...;

CompletableFuture<Person> result = 
  service.findByName("Tom","Schindl");

Let’s for a second suppose we can interact with our UI-Toolkit on ANY-Thread we could simply write:

result.thenAccept( person::set );

Unfortunately none of the UI-Toolkits I know would support this because they require you to sync on a special thread called the UI-Thread. JavaFX is not different in this aspect.

Plain JavaFX APIs:

result.thenAccept( p -> {
   Platform.runLater( () ->
     person.set(p);
   );
} );

Using e(fx)clipse’ ThreadSynchronize#asyncExec(T,Consumer<T>):

import org.eclipse.fx.core.ThreadSynchronize;

// ...

// Lookup the thready service in the service registry
ThreadSynchronize threadSync = 
  ServiceUtils.getService(ThreadSynchronize.class).get();

result.thenAccept( p -> {
   threadSync.asyncExec( p, person::set );
});

We got rid of the inner lambda and replaced it with a method reference, so the code got more readable but

Using e(fx)clipse’ ThreadSynchronize#wrap(Consumer<T>):

// ...
result.thenAccept( threadSync.wrap(person::set) );

we can get rid of all outer lambda as well and are back to fairly the same code as if we’d not had to worry about thread-synchronization at all.

Reading from a worker thread

Let’s suppose you have a Timer running who wants to read a JavaFX-Property from a TextField in 1 second from now you’d:

  • You’d better rewrite it to use a Timeline so that no thread sync is needed
  • You’d write the following magic lines of code:
TextField textField = ...

Timer t = new Timer();
t.schedule( new TimerTask() {
  @Override
  public void run() {
    CountDownLatch ll = new CountDownLatch(1);
    AtomicReference<String> data = new AtomicReference<>();
    Platform.runLater( () -> {
      data.set( textField.getText() );
      ll.countDown();
    });
    ll.await();
    String d = data.get();
    // further process the data 
  }
}, 1000 );

Let’s for a moment forget that this code fairly dangerous because it might create a dead lock situation (hence JavaFX only provides Platform.runLater(Runnable)) this is a huge amount of code to write! Let’s see what APIs e(fx)clipse has to improve the situation.

Using ThreadSynchronize#syncExec(Runnable):

ThreadSynchronize threadSync = 
  ServiceUtils.getService(ThreadSynchronize.class).get();

public void run() {
  AtomicReference<String> data = new AtomicReference<>();
  threadSync.syncExec( () -> {
    data.set( textField.getText() );
  } );
  String d = data.get();
  // further process the data 
}

Removes the need for the CountDownLatch

Using ThreadSynchronize#syncExec(Callable<V>, V):

public void run() {
  String d = threadSync.syncExec( textField::getText, "" );
}

Removes the need for the AtomicReference

Using ThreadSynchronize#scheduleExecution(long, Runnable):

ExecutorService s = ...;
threadSync.scheduleExecution( 1000, () -> {
   String data = textField.getText();
   s.submit( () -> ... );
} );

Removes the thread synchronization problems arising from Platform.runLater() call

Using ThreadSynchronize#scheduleExecution(long, Callable<T>) : CompletableFuture<T> :

threadSync.scheduleExecution( 1000, textField::getText )
  .thenAcceptAsync( d -> ... );

Removes the lambda and gets you to the wonderful CompletableFuture-API.

Block program flow

Generally speaking halting the program flow is a discouraged software pattern and you’d better work with Future and callbacks like Consumer but there might be (existing) API you have to support who requires you to halt the program flow and continue after a certain condition has been met.

To support such a usecase e(fx)clipse has org.eclipse.fx.ui.controls.Util#waitUntil( BlockCondition blockCondition ) you can use like this

Pane p = ...;

// Ask for name in an overlay
String askForName() {
  BlockCondition<String> condition = new BlockCondition<>();

  TextField t = new TextField();
  Button b = new Button("Proceed");
  b.setOnAction( e -> { condition.release(t.getText()); } );
  HBox box = new HBox(
    new Label("Name:"),
    t,
    b);
  box.setManaged(false);
  box.autosize();
  box.relocate( 
    p.getWidth() / 2 - box.getWidth() / 2, 
    p.getHeight() / 2 - box.getHeight() / 2 );
  p.getChildren().add( box );
  return Util.waitUntil( condition );
}


by Tom Schindl at March 23, 2017 01:08 PM

Eclipse Newsletter - Internet of all the Things

March 23, 2017 12:30 PM

The articles feature Eclipse IoT projects including, Eclipse Hono, Eclipse ioFog, Eclipse Kapua, and Eclipse Vorto.

March 23, 2017 12:30 PM

Board Election Results 2017

March 23, 2017 11:00 AM

The Eclipse Foundation announces the 2017 Board Member Election Results.

March 23, 2017 11:00 AM

Orion 14.0 New and Noteworthy

by Mike Rennie at March 21, 2017 07:46 PM

Another three months and another awesome release! Its that time again where I share all of the cool new features, enhancements and fixes with you. As usual with every release, there were lots of changes, so lets jump right in.

Accessibility

The work that began in 13.0 to make Orion completely accessible to every developer continued at a furious pace in 14. This time around, our work was focussed on having the correct colour contrast.

We tightened up our colours in the light theme so that all colours pass the WCAG 2.0 AA guideline for colour contrast. The changes are subtle, but they do make text easier to read, as seen in this before-and-after snapshot of selected code in the editor.

Selected text comparison

Comparing selected text in 14.0 vs. 13.0

Language Tools

Automatic Project Configuration

The JavaScript tooling can now read and understand complex project configurations and automatically configure Tern for the best development experience. For example, the tools can better read and understand package.json files and automatically load available plugins into Tern (rather than the user having to tailor their configuration settings).

Projects Anywhere

Using the new support from the platform to find project contexts, the JavaScript tools can now support a “project” at any level in the navigator. Where a project means any folder that contains JavaScript project-like files – package.json, .tern-project, etc.

Smarter Defaults

The default configuration for the JavaScript tools has been retooled to provide more support right out of the box. In Orion 13.0 (and before), we started the tools in a very bare-bone fashion, and would alert you about potential configuration changes (with quick fixes). Now we automatically start with ECMA, node and browser support, and configure your project as you code.

Disable Linting In-File

Tired of being nagged about a particular code pattern used in certain places (but like to be warned elsewhere)? You can now use the new quickfix to ignore the problem in the current file.

Disable rule in-file

Disable rule in-file

JavaScript Type Icons

In an effort to make the overload of information (while coding in JavaScript) a bit more understandable, we have added icons to help users immediately understand the type of something. For example, F stands for function, O is for objects, C is for classes, etc.

Type icons

Type icons

Improved ESLint configuration file support

We have improved how the JavaScript tools handle the various forms of ESLint configuration files. We now properly support all entries of the files (except for extends).

SVG Support

The CSS and HTML parsers have been updated to properly support SVG attributes and properties. The HTML and CSS validation has also been updated to properly process the new attributes and properties.

Platform Improvements

Syntax Styling

Syntax styling grammars can now define a firstLineMatch attribute.  This enables multiple grammars to be defined for a content type, and the grammar that gets applied will be chosen based on the first line of content.

Tasks

The node server now stores its tasks metadata in a Mongo DB when running as multi-tenant.  As a result, requests querying long-running tasks can now be handled by different server instances that have access to the shared Mongo DB.

Automatic Syntax Checking

Previously, syntax checking took place when a file is saved – if you have autosave turned on in Orion, this is not a problem, as problem markers would be updated as you made changes. If however, you had autosave turned off, any problem markers would quickly become stale causing confusion. Now, in Orion 14, syntax checking will take place on a regular interval even if autosave is turned off, to try and avoid stale problem markers piling up.

New File Client API

The Orion file client has been updated with the ability to find a project given a particular resource path. The new API can be invoked as:

fileClient.getProject(resourcePath, options)

Information Annotations

A new type of annotation has been added to Orion – the info annotation.

The info annotation

The “info” annotation

Annotation Visibility

Always wanted to only show annotations in certain parts of the IDE? Well, now you can.

Simply navigate to the editor settings preference page, and look for the Annotations, Overview Annotations and Text Annotations sections to configure annotation visibility as you’d like.

Annotation visibilities

Annotation visibilities

Don’t forget, you can also use the handy star buttons to have the preference(s) show up in the quick preference menu.

IDE Themes

Finally, after all this time, we have the ability to change the theme of not just the editor, but the entire IDE from the preferences!

Not happy with the default theme in Orion? Head over to the IDE Theme preferences page to change to another theme (currently there are only two of them) or create your own (by modifying an existing theme and saving it as your own).

IDE Theme preferences

IDE Theme preferences


by Mike Rennie at March 21, 2017 07:46 PM

Open IoT Challenge 3.0 — Winners

by Roxanne on IoT at March 20, 2017 10:07 AM

In case you missed the Eclipse IoT announcement last week, the Open IoT Challenge 3.0 winners were announced!

And the winners are…

Open IoT Challenge 3.0 Winners

Congratulations to the InMoodForLife team for coming in first. Their solution analyzes and monitors the sleep patterns of individuals affected by bipolar disorder. The goal of the solution is to improve the therapeutic approach, react, and adapt the treatment faster. The team already shared part of their future plans with us and it sounds very promising! We hope they will go on to help many individuals suffering from bipolar disorder.

The krishi IoT and RHDS teams also worked very hard and delivered great solutions. Read the full announcement to find out more.

Open IoT Challenge 3.0 Sponsors

by Roxanne on IoT at March 20, 2017 10:07 AM

Scala is here

by codepitbull at March 20, 2017 12:00 AM

TL;DR

  • Scala support for Vert.x is here!
  • It is based on Scala 2.12, no support for 2.11 planned
  • all Vert.x-modules are available in a Scala flavor
  • It’s awesome
  • Get started here

Intro

The rise of Scala as one of the most important languages on the JVM caught many (me included) by surprise. This hybrid of functional and imperative paradigms struck a chord with many developers. Thanks to Scala a lot of people who’d never have touched a language like Haskell got exposed to functional programming. This exposure was one of the driving forces to get streams and lambda into the JVM.

With the release of Vert.x 3.4.0 we finally introduced Scala to the family of supported languages: vertx-lang-scala.

In this post I will introduce the new stack and how the power of Scala can be used in your favorite reactive toolkit.

Basics

vertx-lang-scala is based on Scala 2.12. There are no plans to support 2.11.

All modules available for Vert.x are supported (you can check here ).

Future and Promise both need a ExecutionContext
Modules use the following naming-scheme: io.vertx:-scala_2.12:. The Scala version of io.vertx:vert-web:3.4.0 would be io.vertx:vertx-web-scala_2.12:3.4.0.

There is an sbt-based quickstart-project available that will be updated for each Vert.x-release.

Please note: Although sbt is used in this quickstart it is by no means required. There are no special plugins involved so vertx-lang-scala can easily be used with Gradle or Maven.

I use sbt as it is the default build system used for Scala projects.

Quickstart

Let’s get started by cloning the quickstart:

git clone git@github.com:vert-x3/vertx-sbt-starter.git

You just got the following things:

  • An sbt project containing dependencies to Vert.x-core and Vert.x-web
  • The ability to create a fat-jat via sbt assembly
  • The ability to create a docker container via sbt docker
  • A few example verticles
  • Unit test examples
  • a pre-configured Scala-shell inside sbt

We will now run the application to get some quick satisfaction. Use sbt assembly to produce the fat-jar followed by java -jar target/scala-2.12/vertx-scala-sbt-assembly-0.1-SNAPSHOT.jar. Now point your browser to http://localhost:8666/hello for a classic welcome message.

The details

Open your IDE so we can take a look at what’s going on under the hood. We start with the HttpVerticle.

package io.vertx.scala.sbt

import io.vertx.lang.scala.ScalaVerticle
import io.vertx.scala.ext.web.Router

import scala.concurrent.Future

class HttpVerticle extends ScalaVerticle { // <1>


  override def startFuture(): Future[Unit] = { // <2>
    val router = Router.router(vertx) // <3>
    val route = router
      .get("/hello")
        .handler(_.response().end("world"))

    vertx //<4>
      .createHttpServer()
      .requestHandler(router.accept)
      .listenFuture(8666, "0.0.0.0")  // <5>
        .map(_ => ()) // <6>
  }
}
  1. ScalaVerticle is the base class for all Scala-Verticles. It provides all required methods to integrate with the Vert.x-runtime.
  2. There are two ways to start a Verticle. Overriding startFuture, like in this example, tells Vert.x to only consider the Verticle fully started after the returned Future[Unit] has been successfully completed. Alternatively one can override start and by that signal to Vert.x the instant availability of the Verticle.
  3. This block creates a Router for incoming HTTP-requests. It registers a handler to answer with “world” if a request to the URL “/hello” arrives. The class is coming from the Vert.x-web-module.
  4. Every Verticle has access to the Vert.x-instance. Here we use it to create a webserver and register our router to handle incoming requests.
  5. We finally reached the reason why I use startFuture in the first place. All operations in Vert.x are asynchronous. So starting the webserver most definitely means it takes some more time until it bound to the given port (8666 in this case). That’s why listenFuture is used, which returns a Future which in turn contains the actual instance of the webserver that just got started. So our Verticle will be ready to receive requests after the returned Future has been completed.
  6. In most cases we can return the Future directly. In this case the Future returned by listenFuture has the wrong type. We get a Future[HttpServer] but we need a Future[Unit] as you can see in the signature of startFuture. This call takes care of mapping the given Future[HttpServer] to the required return type.

Testing

I use ScalaTest for all my testing needs. It comes with stellar support for asynchronous operations and is a perfect fit for testing Vert.x-applications.

The following HttpVerticleSpec shows how to test an HTTP-API using only Vert.x-classes. Personally I prefer REST-assured with its rich DSL. For this post I wanted to stick with Vert.x-API, so here we go.

package io.vertx.scala.sbt

import org.scalatest.Matchers

import scala.concurrent.Promise

class HttpVerticleSpec extends VerticleTesting[HttpVerticle] with Matchers { // <1>

  "HttpVerticle" should "bind to 8666 and answer with 'world'" in { // <2>
    val promise = Promise[String] // <3>

    vertx.createHttpClient()  // <4>
      .getNow(8666, "127.0.0.1", "/hello",
        r => {
          r.exceptionHandler(promise.failure)
          r.bodyHandler(b => promise.success(b.toString))
        })

    promise.future.map(res => res should equal("world")) // <5>
  }

}
  1. VerticleTesting is a base class for your tests included with the quickstart-project. It’s a small helper that takes care of deploying/un-deploying the Verticle to be tested and manages a Vert.x-instance. It additionally extends AsyncFlatSpec so we can use Futures as test-return-types.
  2. Isn’t it nice and readable?
  3. The promise is required as the whole test will run async
  4. We use the vertx-instance provided by VerticleTesting to create a Netty-based HttpClient. We instruct the client to call the specified URL and to succeed the Promise with the returned body.
  5. This creates the actual assertion. After getting the Future from the Promise an assertion is created: The Result should be equal to the String “world”. ScalaTest takes care of evaluating the returned Future.

That’s all you need to get started!

Futures in vertx-lang-scala

Now for a more in depth topic I think is worth mentioning. vertx-lang-scala treats async operations the Scala-way which is a little different from what you might be used from Vert.x. For async operations like subscribing to the eventbus or deploying a Verticle you would call a method like this:

vertx.deployVerticle("com.foo.OtherVerticle", res -> {
  if (res.succeeded()) {
    startFuture.complete();
  } else {
    startFuture.fail(res.cause());
  }
});

The deployVerticle method takes the Verticle-name and a Handler[AsyncResult] as its arguments. The Handler[AsyncResult] is called after Vert.x tried deploying the Verticle. This style can also be used for Scala (which might ease the transition when coming from the Java-world) but their is a way more scalaish way of doing this.

For every method taking a Handler[AsyncResult] as its argument I create an alternative method using Scala-Futures.

vertx.deployVerticleFuture("com.foo.OtherVerticle") // <1>
  .onComplete{  // <2>
    case Success(s) => println(s"Verticle id is: $s") // <3>
    case Failure(t) => t.printStackTrace()
  }
  1. A method providing a Future based alternative gets Future appended to its name and returns a Future instead of taking a Handler as its argument.
  2. We are now free to use Future the way we want. In this case onComplete is used to react on the completion.
  3. Pattern matching on the result <3

I strongly recommend using this approach over using Handlers as you won’t run into Callback-hell and you get all the goodies Scala provides for async operations.

Future and Promise both need a ExecutionContext
The VertxExecutionContext is made implicitly available inside the ScalaVerticle. It makes sure all operations are executed on the correct Event Loop. If you are using Vert.x without Verticles you have to provide it on your own.

Using the console

A great feature of sbt is the embedded, configurable Scala-console. The console available in the quickstart-project is pre-configured to provide a fresh Vert.x-instance and all required imports so you can start playing around with Vert.x in an instant.

Execute the following commands in the project-folder to deploy the HttpVerticle:

sbt
> console
scala> vertx.deployVerticle(nameForVerticle[HttpVerticle])
scala> vertx.deploymentIDs

After executing this sequence you can now point your browser http://localhost:8666/hello to see our message. The last command issued shows the Ids under which Verticles have been deployed.

To get rid of the deployment you can now type vertx.undeploy(vertx.deploymentIDs.head).

That’s it!

This was a very quick introduction to our new Scala-stack. I hope to have given you a little taste of the Scala goodness now available with Vert.x. I recommend digging a little more through the quickstart to get a feeling for what’s there. In my next blog post I will explain some of the decisions I made and the obstacles I faced with the differences between Java and Scala /Hint: They are way bigger than I was aware of).

Enjoy!


by codepitbull at March 20, 2017 12:00 AM

Eclipse Demo Schedule at Devoxx US

March 19, 2017 10:40 AM

Visit the Eclipse Foundation at booth #318 on March 21-23 in San Jose, CA for some exciting demos.

March 19, 2017 10:40 AM

Announcing Orion 14

by Mike Rennie at March 17, 2017 06:56 PM

We are pleased to announce the fourteenth release of Orion, “Your IDE in the Cloud”. You can run it now on OrionHub or download the server to run your own instance. Once again, thank you to all committers and contributors for your hard work this release.  There were 150 bugs and enhancements fixed, across more than 380 commits from 14 authors!

What’s new in Orion 14?  This release was focussed on quality and ease of use – Orion 14 is more accessible (better colours and accessibility), easier to start coding in (the tools now automatically understand complex project configurations, so you don’t have to), and just more awesome in general.

We continued to improve the Node.js server (which is used on orion.eclipse.org or locally), and continued to improve our Electron app. Lastly, we began work in 14.0 to provide collaborative development support and debugging support directly in Orion! Stay tuned in Orion 15 for these features to officially land.

Enjoy!


by Mike Rennie at March 17, 2017 06:56 PM

Debugging Xtext grammars – what to do when your language is ambiguous

by Holger Schill (schill@itemis.com) at March 17, 2017 03:00 PM

Xtext uses ANTLR to generate a lexer and parser out of your grammar. Technically an LL(*) parser gets generated. This means it cannot deal with left recursion and has an infinite lookahead. You might know what that means, but to make it easier you could think about LL(*) parsers like this: A parser gets an ordered list of things (called tokens) to collect in a labyrinth. When it’s not clear which way to go it stands still and tries to look in all directions until their end. As soon as it is obvious where to go, it continues walking and collecting. There is no way back – so decisions should be correct. Sometimes this is not the case and the parser can make a clear decision. In this situation it gets tricky to understand where the problem is and how to resolve it. Mostly shown errors and warnings are not that meaningful.

debugging-xtext-grammar.jpg


What do you normally do when the Xtext workflow reports warnings or errors while the parser gets generated? Obviously errors can’t be ignored since the parser will not get generated and the workflow fails – but what about warnings? Do you try to solve them by staring at the grammar and try to think like a parser? Or you ignore them because it seams to work? Really?

In projects we have seen people dealing with such problems in various ways. Ignoring warnings is not a good idea, since ANTLR switches off alternatives and you do not know which one. We have seen people consequently ignoring such warnings, because they cannot figure out the real cause and things got complex. However ignoring those warnings should not be an option.

I have seen that a large group of Xtext users do not know ANTLRWorks or that it can help here. So let’s make two trivial examples to see how to use the tool with Xtext.

Warnings

Let’s make a trivial example where it is really obvious what the problem is:

example-grammar-debugging-xtext-grammars.png

We have two parser rules (Element1 and Element2) that look identically except that there are different parts optional. During the workflow runs it reports the following warnings:

warning(200): ../com.itemis.blog.antlrworks.dsl/src-gen/com/itemis/blog/antlrworks/parser/antlr/internal/InternalDsl.g:114:2: Decision can match input such as "'element' 'id' RULE_ID 'int' RULE_INT" using multiple alternatives: 1, 2

As a result, alternative(s) 2 were disabled for that input

warning(200): ../com.itemis.blog.antlrworks.dsl.ide/src-gen/com/itemis/blog/antlrworks/ide/contentassist/antlr/internal/InternalDsl.g:156:1: Decision can match input such as "'element' 'id' RULE_ID 'int' RULE_INT" using multiple alternatives: 1, 2 

In this case the parser gets generated, but it tells us that there were different alternatives for the same input and that ANTLR decided to disable 2 of them. It does not tell us which once. Let’s find out what the cause is.

ANTLRWorks comes as an executable jar – running it should not be a problem as long as java is installed. If you want to open a grammar it expects an *.g file. Xtext should have generated one in the src-gen folder. In our example there is a .g file located here: com.itemis.blog.antlrworks.dsl/src-gen/com/itemis/blog/antlrworks/parser/antlr/internal/InternalDsl.g

The grammar looks a bit strange and there is a lot of Java stuff in the grammar. ANTLRWorks will fail to compile the grammar… damn.

Error-debugging-xtext-grammars.pngOk, the generated ANTLR grammar cannot be directly used in ANTLRWorks since it is modified to the needs of Xtext. To generate a so-called debugable grammar you need to modify the workflow a bit like this.

workflow-debugging-xtext-grammars.png

Now you’ll find a .g file in src-gen that carries the name DebugInternal*.g. This file can be easily used with ANTLRWorks.

Path-debugging-xtext-grammars.png

After you have started ANTLRWorks click on File->Open and select the DebugInternal*.g file.

File-debugging-xtext-grammars.png

ANTLRWorks will open the grammar and you’ll see the different rules. So far no warnings are shown. To let the tool do it’s job click on debug – the button looks like a bug. After doing that the ruleElement is marked read. By clicking on the rule you’ll see the problem that caused the warnings and the different alternatives. To really see what the disabled alternatives are you could enable them as shown in the next picture. The read arrows will show the disabled alternatives.

ANTLRWorks-debugging-xtext-grammars.pngAs already said this is a very trivial example and the tool just points out what we already know. In typical projects we have far more complex scenarios where it is nearly impossible to get the cause of a warning without ANTLRWorks. Especially when a lot of parts are optional it gets tricky. To really understand what the parser does there is the possibility to debug the grammar with a given input and see how the parsetree is constructed. Clicking on debug once more brings up a window to define the input.

Input-debugging-xtext-grammars.png

Debugging in ANTLRWorks works similar like you know it from Eclipse and you can step forward and backward. At the end the parsetree will show that the parser went into the rule “Element1” instead of “Element2”. From a grammar point of view both rules would be valid but ANTLR switched of the alternative. Otherwise no clear decision could be made.

debugging-xtext-grammars.png

If we add a second line as an input for the debugger and leave out the intValue the parsetree looks like this:

Input-debugging-xtext-grammars.png

debugging-xtext-grammars.png

As the intValue is mandatory in the ruleElement1 the parser will go into ruleElement2 for the second entry. This is the only case where “Element2” is picked. In this trivial example it’s not that hard to guess what the parser will do. In more complex example this debugging feature will really bring a big benefit to solve your ambiguities. 

Errors

 What about errors? Do you know what to do when the workflow reports that a rule has a non-LL(*) decision? What the hell is left-refactoring, syntactic predicates and why should I use backtracking – should I really? We’ll handle that in another blogpost, but for now let’s have a look at a simple language that has some expressions.

grammar-debugging-xtext-grammars.png 

When we try to run the Xtext workflow the generator will report the following error:

 error(211): ../com.itemis.blog.antlrworks.dsl/src-gen/com/itemis/blog/antlrworks/parser/antlr/internal/InternalDsl.g:114:2: [fatal] rule ruleExpression has non-LL(*) decision due to recursive rule invocations reachable from alts 1,3. Resolve by left-factoring or using syntactic predicates or using backtrack=true option. 

Various exceptions are show below the error, but Xtext will generate the .g file anyway – so there is a chance to find out what the problem is. You might already know what’s wrong, but let’s try to use ANTLRWorks. The compiler in ANTLRWorks will show the very same error, but after ignoring that the rule element is marked red. The different alternatives are marked red.

ANTLRWorks-Error-debugging-xtext-grammars.png
In this case left-recursion is not our problem. ANTLRWorks shows us, that ruleBlockExpression and ruleListLiteral are the cause.

rule1.png

rule-debugging-xtext-grammars.png

After having a closer look it is obvious that the syntax is equal if there is only one expression inside – that makes our grammar ambiguous. Do we really want a ListLiteral to exist on the same level as a BlockExpression? Do we really want a BlockExpression contain various other BlockExpressions?

After considering these questions a refactored grammar looks like this:

refactored-grammar-debugging-xtext-grammars.png

After doing that the workflow will run through and we are good to have a second look in ANTLRWorks to see the parsetree for a simple expression:

input-debugging-xtext-grammars.png

debug-Expression-debugging-xtext-grammars.png

These examples are very trivial to make it obvious where the problem is. The intension was to let you know how to use ANTLRWorks with Xtext. Don't ignore warnings anymore – you might not know the implications. The parsetree might look different as you though.

Stay tuned for another post about syntactic predicates, left-recursion / left-refactoring and why backtracking is not an option. And if you've got any questions regarding Xtext – don't hesitate to contact us!

Contact the  itemis Xtext team


by Holger Schill (schill@itemis.com) at March 17, 2017 03:00 PM

Vert.x and IoT in Rome : what a meetup !

by ppatierno at March 17, 2017 10:45 AM

Yesterday I had a great day in Rome for a meetup hosted by Meet{cast} (powered by dotnetpodcast community) and Codemotion, speaking about Vert.x and how we can use it for developing “end to end” Internet of Things solutions.

17352445_10208955590111131_6229030843024604532_n

17352567_10208955588791098_766816304298598626_n

I started with an high level introduction on Vert.x and how it works, its internals and its main usage then I moved to dig into some specific components useful for developing IoT applications like the MQTT server, AMQP Proton and Kafka client.

17342690_10208955588751097_8818320599257580571_n

17352571_10208955588951102_2851165399929439718_n

It was interesting to know that even in Italy a lot of developers and companies are moving to use Vert.x for developing microservices based solutions. A lot of interesting questions came out … people seem to like it !

Finally, in order to prove the Vert.x usage in enterprise applications I showed two real use cases that today work thanks to the above components : Eclipse Hono and EnMasse. I had few time to explain better how EnMasse works in details, the Qpid Dispatch Router component in particular and for this reason I hope to have a future meetup on that, the AMQP router concept is quite new today ! In any case, knowing that such a scalable platform is based (even) on Vert.x was a great news for the attendees.

17264802_10208955590191133_8923182437405273553_n

If you are interested to know more about that, you can take a look to the slides and the demo. In the coming days, the video of the meetup will be available online but it will be in Italian (my apologies for my English only friends :-)). Hope you’ll enjoy the content !

Of course, I had some networking with attendees after the meetup and … with some beer 🙂

17310150_1421561734583219_8414988688301135801_o



by ppatierno at March 17, 2017 10:45 AM

JBoss Tools 4.4.4.AM1 for Eclipse Neon.2

by jeffmaury at March 15, 2017 09:50 PM

Happy to announce 4.4.4.AM1 (Developer Milestone 1) build for Eclipse Neon.2.

Downloads available at JBoss Tools 4.4.4 AM1.

What is New?

Full info is at this page. Some highlights are below.

OpenShift 3

Although our main focus is bug fixes, we continue to work on providing better experience for container based development in JBoss Tools and Developer Studio. Let’s go through a few interesting updates here and you can find more details on the What’s New page.

OpenShift Server Adapter enhanced flexibility

OpenShift server adapter is a great tool that allows developers to synchronize local changes in the Eclipse workspace with running pods in the OpenShift cluster. It also allows you to remote debug those pods when the server adapter is launched in Debug mode. The supported stacks are Java and NodeJS.

As pods are ephemeral OpenShift resources, the server adapter definition was based on an OpenShift service resource and the pods are then dynamically computed from the service selector.

This has a major drawback as it allows to use this feature only for pods that are part of a service, which may be logical for Web based applications as a route (and thus a service) is required in order to access the application.

So, it is now possible to create a server adapter from the following OpenShift resources:

  • service (as before)

  • deployment config

  • replication controller

  • pod

If a server adapter is created from a pod, it will be created from the associated OpenShift resource, in the preferred order:

  • service

  • deployment config

  • replication controller

As the OpenShift explorer used to display OpenShift resources that were linked to a service, it has been enhanced as well. It now displays resources linked to a deployment config or replication controller. Here is an example of a deployment with no service ie a deployment config:

server adapter enhanced

So, as an OpenShift server adapter can be created from different kind of resources, the kind of associated resource is displayed when creating the OpenShift server adapter:

server adapter enhanced1

Once created, the kind of OpenShift resource adapter is also displayed in the Servers view:

server adapter enhanced2

This information is also available from the server editor:

server adapter enhanced3

Server Tools

API Change in JMX UI’s New Connection Wizard

While hardly something most users will care about, extenders may need to be aware that the API for adding connection types to the &aposNew JMX Connection&apos wizard in the &aposJMX Navigator&apos has changed. Specifically, the &aposorg.jboss.tools.jmx.ui.providerUI&apos extension point has been changed. While previously having a child element called &aposwizardPage&apos, it now requires a &aposwizardFragment&apos.

A &aposwizardFragment&apos is part of the &aposTaskWizard&apos framework first used in WTP’s ServerTools, which has, for a many years, been used throughout JBossTools. This framework allows wizard workflows where the set of pages to be displayed can change based on what selections are made on previous pages.

This change was made as a direct result of a bug caused by the addition of the Jolokia connection type in which some standard workflows could no longer be completed.

This change only affects adopters and extenders, and should have no noticable change for the user, other than that the below bug has been fixed.

Forge Tools

Forge Runtime updated to 3.6.0.Final

The included Forge runtime is now 3.6.0.Final. Read the official announcement here.

startup

Enjoy!

Jeff Maury


by jeffmaury at March 15, 2017 09:50 PM

Papyrus-IC Research/Academia webinar

by eposse at March 15, 2017 05:19 PM

On Friday, March 17th at 16:00 CET, 15:00 GMT, and 11:00 EDT, the Papyrus Industry Consortium’s (a.k.a. Papyrus-IC or, as I prefer, Me-IC) Research and Academia committee will host their third webinar of the year. The topic is on an industry perspective on software product lines with speakers from Saab and Pure-Systems. See this link for the connection information.


Filed under: Uncategorized

by eposse at March 15, 2017 05:19 PM

EMF Support for Che – Day 4: Building Che

by Maximilian Koegel and Jonas Helming at March 15, 2017 01:53 PM

In this blog series, we share our experiences extending Eclipse Che to add EMF support.. The first post covers our goals. In previous posts, we describe how to add support for code generation and create a custom stack, which provides the framework for code generation out of the box.

So far, we have not yet written any code to extend Che. We have used the concept of Che workspaces, which are Docker containers, to deploy additional tools (in our case the code generator). As the browser IDE allows to execute any command on the workspace runtime, we did not have to implement something to extend it so far.

However, there are requirements, where enhancing the workspace is not sufficient. This is typically the case when you enhance the UI of the browser IDE with new features or you add new APIs into agents running within workspaces.

In this post, we will describe how to make a minor enhancement to the browser IDE with a simple “hello world” example. It is a good  first step to introduce the general process for extending Che. We will do more complex extensions in future blog posts.

We at EclipseSource have many years of experience in building extensions for the Eclipse IDE and in building Eclipse-based applications. So before we go into detail, let us summarize our experience in developing extensions for Che from a viewpoint of a desktop Eclipse IDE developer. We focus on the most obvious similarities and differences.

The similarity: Eclipse Che has an inherently extensible and flexible architecture. This is mainly due to a central pattern: Service-Orientation. There are several ways how this pattern is used for extensibility:

  • Che provides services to build up almost anything in the UI. As an example, there is a service to register the actions which are shown in the various menus. As another example, there is a service to register file extensions (defining icons and default editors). These services can be layered into complex UI objectives such as creating a perspective with different menus, panels, and layouts.
  • Che provides services to access resources (e.g. the source files) and the workspace runtime (e.g. to trigger commands). By using these existing features, it is fairly simple to add new features on top.
  • The Che server is mainly a collection of RESTful services. By adding custom REST services to the server, you can easily enhance it. Thus custom services can then be consumed by custom extensions in the browser IDE.
  • Finally, Che defines service interfaces, which can be implemented by an extension to provide new things. As an example, you can implement a service, which implements the behavior to create a custom project type.

The difference: There is currently no runtime plugin mechanism for the Che browser IDE comparable to what you are used to from the classic Eclipse IDE (a.k.a update sites/p2 repositories). This means, to extend Che, you need to build your custom version of it which contains your extensions and then deploy the full assembly somewhere. Technically, “plugins” in Che are maven modules, that you add to the global build during compile time. There is no runtime extensibility of the browser IDE. So, isn’t this a step backwards compared to the classic Eclipse IDE? There are different answers to this question, depending on the point of view:

  • No, because Che supports extensibility at runtime based on its workspace concept. If you miss any tool or runtime component, you can simply deploy it into your workspace and easily share it with co-workers. As an example, we enabled the EMF code generation only by extending the workspace at runtime (see this post). In the classic Eclipse IDE, everybody had to install the same things again (at least when not using Oomph).
  • Partially, as you cannot extend neither the browser IDE, nor the Che server at runtime, but only the workspace. However, Che has a different deployment scenario. As it is a client-server application, the idea is to do the set-up once in a central way and then share it for a group of developers, which require the same set of tools. This enables a very fast setup for developers on a project (pretty much like Oomph for classic Eclipse). It even enables to use different IDEs, which access the same workspace. However, it means more work for the author and maintainer of extensions, but a simpler life/set-up for the developers.
  • Yes, we miss OSGi, extension points and p2 repositories! Although OSGi and p2 have been criticized in the past, they are a very powerful combination to build up modular and extensible applications. This in combination with the great tooling provided by Eclipse made it possible to efficiently develop, install, update and deploy extensions to the IDE. This was probably one of the core factors to create such a huge ecosystem of tools.

So at the end, it really depends on the scenario and your design objectives. It is worth mentioning, that there is a general trend (also followed by Che) to move UI related parts of the IDE into server-side abstractions, so that they become independent. The language server protocol is a good example for this. In this scenario, an IDE only has to support to interpret the abstractions, e.g. the LSP. Therefore, client-site extensibility may become less important.

Anyways, let us get started with building Che locally.

 

First, you need install all the prerequisites to build and run Che. We already mentioned that you will need Docker, but there are a number of tools and environment settings needed for building Che.

There are two ways to get started. We recommend to start headless, first, and follow the guide how to clone and build Che. As a second step, in case you want to use an IDE, we recommend the Che workspace setup guide for an extensive overview.

We recommend to take the time to read the provided documentation carefully, as it contains useful information and lots of useful hints how to develop for Che. You should pay special attention to the “Super Dev Mode”, which allows hot code replacement for GWT applications and therefore drastically reduces the turnaround time when working on the browser IDE.

There is also an option to build Che within a preconfigured Docker environment which will spare you the trouble of setting it up yourself. Finally, the building Che will take some time to complete even if you have a fast machine. Therefore, we recommend to take a close look at the options, which are described in detail here.

The build process will result into a number of artifacts, also called assemblies in Che terminology. You can find them within the “assembly” subdirectory. You can match the assemblies to the logical components of the Che architecture. Just compare the directory with the architecture schema and the modules overview.

After the build completed you can start Che on your local machine. The simplest way to do that is to navigate to the assembly/assembly-main/target/[che-version]/[che-version] directory and execute “bin/che.sh start”. A number of long messages should confirm the successful start of the services and point you to open localhost:8080 in your browser. If everything went well you will be greeted by the Che dashboard.

So now that we can build check locally, let us do a very simple change to verify the build process. An example for such a simple change would be to add a project template to the browser IDE. Project templates can be instantiated by any user of the IDE. As we are working on EMF support, it would be useful to have an example EMF project as a template. We have already manually imported such a template from a Git repository in the second part of this blog series, which we will now add as a fixed template to our custom assembly.

Project templates are basically pointers to existing Git repositories. This enables to easily maintain the templates without re-distributing the IDE itself. The sample templates are maintained in the following file:

ide/che-core-ide-templates/src/main/resources/samples.json

Let us add following snippet to this file:

  {
   "name": "emfforms-makeithappen-blank",
   "displayName": "emfforms-makeithappen-blank",
   "path": "/",
   "description": "EMFForms, make it happen!",
   "projectType": "java",
   "mixins": [],
   "attributes": {
     "language": [
       "java"
     ]
   },
   "modules": [],
   "problems": [],
   "source": {
     "type": "git",
     "location": "https://github.com/eclipsesource/emfforms-makeithappen-blank",
     "parameters": {}
   },
   "commands": [],
   "links": [],
   "category": "Samples",
   "tags": [
     "maven",
     "java"
   ]
 }

Afterwards we need to stop the currently running che instance by executing “bin/che.sh stop”, rebuild Che and afterwards start it again using “bin/che.sh start”.

As a result, the new project template will be available among the existing ones:

Of course, this was a very simple change, it did not even involve any coding. However, we are now prepared to do more complex changes and start coding. Please note, that in our example, we changed a configuration file of Che to add our custom project template. Those kind of extensions are usually done by invoking services, which allow to extend the base configuration of Che (in our case add a new project template). We will get back to this cleaner solution later in this series.

When starting to code extensions for Che, those are typically placed in separated maven modules (i.e. plugins). This is conceptually pretty much like developing plugins for classic Eclipse. That means, the custom code will be separated from the core of Che. We will describe this more in detail in the next blog post of this series. As an example, we will create plugin, which registers a custom file type for “.ecore” including a custom icon.

So stay tuned!

Please note, that due to Eclipse Converge and Devoxx US, the next post will be published in 3 weeks. Please note that we will give a talk at Eclipse Converge about our experience with extending Che. So in case you haven’t already, please register soon.
If you are interested in learning more about the prototype for EMF support, if you want to contribute or sponsor its further development, or if you want support for creating your own extension for Che, please feel free to contact us.

_MG_4540b2
Co-Author
Mat Hansen


TwitterGoogle+LinkedInFacebook

Leave a Comment. Tagged with che, eclipse, che, eclipse


by Maximilian Koegel and Jonas Helming at March 15, 2017 01:53 PM