A story of CI build monitoring and creativity at play.

Background

Ten years ago, I had an idea. I wanted to use a Lego Mindstorms NXT 2.0 kit to build a robot that could continuously monitor our software development builds. The following is the story of what happened and how things worked out. Along the journey there were failures, successes, lessons learned and a lot of joy and smiles.

Who is responsible for monitoring broken builds?

When a software development build breaks, who is responsible? The obvious answer is the developer who broke the build. Another answer is the entire development team. Who is responsible for letting the team know that the build is broken? One answer to this question has been to use “automation” to shorten the feedback loop. Continuous Integration (CI) builds run any time a commit is merged to the main branch or on merge quests. When these builds break, how does the team discover that a break has happened? The team would need to continuously monitor the build to know that something has happened.

Some development teams use visual and visible cues to keep the team alerted to problems in the build. These visible cues include solutions like attaching large build monitors to shared wall spaces where the team can look to see the current status of important builds. The problem with a build monitor is that it requires line of sight for all team members. I once sat in a team space where a build monitor was located in an office window right behind the cubicle I was sitting in. So if the build broke, I wouldn’t know, as my back was facing the monitor.

There are other options like using a Build Light Indicator of some sort. All the options I had researched suffered from two problems — requiring line of sight or requiring a physical hardware connection. The reason a physical hardware connection was a problem was because the USB ports on all of our machines at work were disabled due to security policies. We couldn’t plug in a lava lamp or any other light indicator.

I had this crazy idea that my team could build a robot to monitor our builds. The robot would have to look at a screen and alert us when the build breaks. I went to my nearest Lego store and bought a Lego Mindstorms NXT 2.0 kit.

Team Building Exercise

I initially thought that building a Lego Mindstorms robot would be a good team building exercise. Things did not go exactly as I expected. My team started out attempting to build the Lego Mindstorms standard Shooterbot design during some lunch time meetings. We ordered take out food so we could multitask and eat while we built. The first thing I learned was not to mix lunch and lego building. We did have fun eating lunch and even made it through a few pages of the lego building instructions, but it was clear that it was going to take many meetings to finish the robot at this pace.

I decided there was an easier way forward.

I asked my seven year old son if he would like to build the robot at home. Lego Robot + Seven Year Old = Yes. He was finished with the standard build for the Shooterbot in two and a half hours. I brought the finished robot to work and began working out how we could get the robot to monitor the build.

Vision for my vision

I originally had planned to use the Lego Mindstorms Color Sensor to detect color changes on a screen. Unfortunately, the Color Sensor could only really detect colors printed on some solid surface like paper. I tested out the sensor on different monitors, and it wouldn’t work.

I did some research and discovered there was a third party sensor market for Lego Mindstorms. There was a Vision Subsystem option for the NXT 2.0. So I spent some more money and bought the Vision Subsystem and hoped it would work. It worked great.

From Visual Programming to Java

Now I needed to program the robot. At first, I experimented with the visual programming environment that came with the Lego Kit. I was able to control a lot of the sensors and make the robot move, but for more complex tasks it felt like a non-visual programming environment would be a much more productive option. After a bit of searching I discovered the leJOS JVM that could be loaded on the Lego brick and would allow Java code to control the robot.

A member of my team at the time (Craig Motlin) wrote around three hundred lines of Java code to keep the robot continuously monitoring our builds. It was a lot of fun seeing the robot shooting plastic balls when it detected red builds on the screen of my MacBook Air.

Bang a Gong!

Now we had a shooter robot that could detect build failures as well as build success and shoot plastic balls out at high velocity. It was great fun to watch, but was missing a key component that I was looking for. An audible alarm to go along with the visual and physical queues that a build was broken. I was sitting at home one night puzzling about what we could do to have the shooter robot make a noise that could be heard all around a floor.

Then it hit me. Sitting on my desk was a Feng Shui Desktop Gong that my kids had bought me. Along with the gong came a little mallet with a plastic ball attached to the end that was the same size as the plastic balls that the shooter robot was loaded with. Light bulb!

The noise that the gong made when the shooter robot shot the plastic ball and struck the metal gong was loud and piercing. It was so loud and piercing it could be heard all over a huge and noisy exhibit floor at a conference (spoiler alert!).

Measuring Success in Smiles

For a few years, BilBo the Build Bot was one of the simplest and most smile inducing demos around. BilBo had one clear Key Performance Indicator (KPI) — Smiles.

Anyone who saw BilBo when visiting or walking by our team area asked for a demo. Each demo took less than 30 seconds after the Lego brick was powered up, or 5–10 seconds if it was already running.

Power Failure

One problem that persisted with Bilbo was that it was powered by four AA batteries, which had a lifespan of several hours if left running. I did some research on a persistent power source for BilBo but was unable to find anything that I thought would work. It turns out the BilBo demos were so amazingly popular, it wasn’t so terrible that we couldn’t find a persistent power source option. We were able to generate hundreds of smiles using just four AA batteries at a time.

BilBo at JavaOne 2014

In 2014, I brought BilBo to San Francisco with me to demo at a booth that my former employer had at JavaOne. BilBo was running Java on the brick so was a great demo at the conference.

Only the Beginning

Nikhil Nanivadekar would attend his first JavaOne in 2015. He would later evolve the shooter robot idea and turn it into another smile inducing demo with two Twitter controlled robots at JavaOne 2016. Nikhil was building using the upgraded Lego Mindstorms EV3 kit. We would give a talk twice at JavaOne 2016 titled “Robots for the Kind in All of Us”. There were many more smiles measured.

For JavaOne 2017, Nikhil would build a small army of standing shooter robots that would take aim at six of the Desktop Gongs.

The robots were amazing, but it was arguably the gong that was the key to success for all of the demos. The gong gave the robots a goal to achieve, and an audible way to make people smile. Anyone who witnessed the demo, knew immediately on the exhibit floor, regardless of where they were at the time, what was happening when they heard the gongs — robots were banging gongs and making people smile. Just hearing the noise would make folks who were “in the know” smile :)

I hope this blog inspires some great ideas that brings more smiles to the world.

Thank you for reading, and hopefully smiling! :)

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Europe’s leading open source foundation continues its momentum adding multiple new members as well as seven new projects for building an ecosystem for the software-defined vehicle 

BRUSSELS – June 30, 2022 – The Eclipse Foundation, one of the world’s largest open source foundations, today announced major momentum with seven new projects initiated with the Software Defined Vehicle (SDV) Working Group, as well as a growing member base focused on innovation for the software-defined vehicle ecosystem. In just a few months, the SDV Working Group has expanded its membership by more than 30% to 19 members, including new members Volkswagen Group’s software unit CARIAD, Eteration, FutureWei, NXP, and Toyota. 

Using a “code first” approach, the SDV-related projects are focused on building the industry’s first open source software stacks and associated tooling for the core functionality of a new class of automobile. The SDV Working Group believes this approach will result in much faster material impact on the industry. These new projects, headed up by leaders such as Bosch, Microsoft, Continental, ZF, Cariad, Accenture, and Eteration, are already making their software available to any organization that wishes to leverage them for their own vehicle development. 

“I’m thrilled with the industry response to our Software Defined Vehicle Working Group,” said Mike Milinkovich, executive director of the Eclipse Foundation. “In just a few short months, we’ve had seven projects contributed, which will collaboratively develop real code for all aspects of the automotive software ecosystem. We expect this momentum will only increase in the coming months.”

To support this transformation to software-defined vehicles, major players from the broader technology industry, as well as the automotive industry, are actively developing an open source in-vehicle application runtime stack, cloud-based vehicle operations, as well as highly integrated development toolchains. The open source Software Defined Vehicle initiative aims to deliver usable open source code for in-vehicle software across vehicle models, product lines, brands, organizations, and time. Doing so enables an order of magnitude increase in the speed of innovation, speed of production, and the ability to scale the production of software-centric vehicles. What’s more, this approach will significantly reduce the complexity of designing new vehicles, while also increasing efficiency. The ultimate goal is that industry players will benefit by being able to focus on innovation, while saving time and cost on the non-differentiating elements, like real-time operating systems, specific parts of the middleware layers or communication protocols.

Some of the new Software Defined Vehicle-related projects include:

Eclipse Velocitas

Eclipse Velocitas provides an end-to-end, scalable, modular, and open source development toolchain for creating containerized and non-containerized in-vehicle applications.

Eclipse Muto

Eclipse Muto provides an adaptive framework and a runtime platform for dynamically composable model-driven ROS software stacks on autonomous vehicles and robots in general. Eclipse Muto makes it easier to build an adaptive software stack that is able to update the system without compromising its mission, plan, safety, and security while it is active. Eclipse Muto can be used to introspect, monitor, and manipulate the actively running ROS graph (the network of ROS nodes) with a lightweight model and digital twin.

Eclipse eCAL (enhanced Communication Abstraction Layer)

The Eclipse eCAL (enhanced Communication Abstraction Layer) provides a middleware solution that enables scalable, high-performance inter-process communication on a single computer node or between different nodes in a computer network. Eclipse eCAL supports a wide range of hardware targets, operating systems, and programming languages.

Eclipse eCAL is shipped with sophisticated tools for live data monitoring, orchestrated message recording, and replay. It’s designed for rapid prototyping in the field of autonomous driving and has already proven its performance over the past few years in research projects of Continental as well as in other open source community projects.

Eclipse Leda

One of the main challenges in building open software for the SDV will be the combination of the diverse components into a coherent and useful whole; all the software components in the world will not have the impact needed to transform the automotive industry unless we can make them play together coherently and form a functional portfolio.

As a first step towards that goal, Eclipse Leda provides an “SDV distribution” that pulls together individual contributor pieces from the SDV WG and the larger OSS community to deliver a functional and always-available Linux-based image/distribution.

Eclipse Chariott

The Eclipse Chariott project aims to simplify and enhance in-vehicle software developer productivity by providing a metadata-driven middleware/abstraction layer that allows modern application programming models to target in-vehicle functions through a digital representation of vehicle state and capabilities and provides an extensible and dynamic architecture to access the vehicle hardware and sensors.

Eclipse ADAAA (Adaptive Cruise Control Demo Application for Adaptive AUTOSAR)

ADAAA is an example application for Adaptive AUTOSAR with the following goals:

1. Provide a tutorial with simple example code bases for getting involved with the Adaptive AUTOSAR methodology and each functional cluster
2. Use the Adaptive Cruise Control function to demonstrate the use of Adaptive AUTOSAR
3. Build Adaptive AUTOSAR expertise across the community
4. Use the community support to adapt to different Adaptive AUTOSAR provider

Eclipse SommR

Eclipse SommR provides an automotive-grade implementation of the someIP specification for embedded Linux systems together with the required tools to support developers.

The Eclipse Foundation has decades of experience managing the governance of complex technology initiatives and multi-vendor organizations, making it the ideal organization to help manage such an endeavor. Its commitment to transparency, vendor-neutrality, and a shared voice will ensure that all participants have the opportunity to shape the future of the working group.

To learn more about getting involved with the Eclipse Foundation’s Software Defined Vehicle initiative, please visit us at sdv.eclipse.org, or email us at membership@eclipse.org. 

Quotes from Members

Bosch

“We are excited to see the first projects contributed to the SDV working group as this is a big milestone and underlines the “code first” approach,” said Sven Kappel, VP and Head of Project Software Defined Vehicle, Bosch. “Additionally, we are also proud that we, as Bosch, are contributing two projects: Eclipse Velocitas and Eclipse Leda. We are looking forward to developing the projects together with the community.”

CARIAD

“As one of the leading software companies in the automotive industry, we are looking forward to the open cooperation in the Eclipse Foundation,” said Michael Wintergerst, executive vice president for Vehicle and Cloud Platform, CARIAD. “Open source has long been a central component in the automotive sector. We see great potential in the cooperation of the open source community for the software-driven car and also for our developers. We want to create innovations and set open standards together.”

Continental Automotive 

“We are glad to support the ‘code first’ approach of Eclipse SDV by contributing Eclipse eCAL: a scalable middleware enabling efficient communication within distributed systems,” said Michael Huelsewies, Head of Architecture and Software at Continental Automotive. “Continental is looking forward to collaborating within the community to create innovative solutions for the Software-Defined Vehicle.”

ETAS

“The first Eclipse SDV Contribution Day sets a major milestone on the road to an open-source software-defined vehicle ecosystem: first, we look forward to listening to the community's projects and are excited to present our two projects. Second, we welcome the growth of the community and the addition of new participants building an ecosystem for the software-defined vehicle,” said Christoph Hartung, Chairman of the Board of Management of ETAS GmbH. “We are convinced that cross-company collaboration will lead to excellent results.” 

Eteration

“Industry practices for automotive software often embrace proprietary and closed solutions, where many common building blocks are replicated. The Eclipse Software Defined Vehicle Working Group is a platform for open technology and collaboration that can impact how we build software for vehicles,” said Naci Dai, founder, Composiv.ai and Eteration. “Eteration is contributing Eclipse Muto a high-performance runtime for model based contextual adaptivity in ROS software stacks. Eclipse Muto makes it possible to deploy extensible, dynamic, and adaptive behavior to edge devices.”

Microsoft 

“The Software Defined Vehicle Working Group provides a vendor-neutral place for software engineers to jointly solve non-differentiating challenges of automotive software development,” said Ulrich Homann, corporate vice president and Distinguished Architect, Cloud + AI, Microsoft. “Microsoft is contributing the middleware ‘Eclipse Chariott’ to target in-vehicle functions through a digital representation of vehicle state and capabilities. The dynamic architecture enhances developer productivity, freeing resources to focus on delivering new services that address latest customer needs.”

ZF

“We are pleased to welcome the participants of the first Contribution Day in Friedrichshafen and fully support the open source approach of the new Software Defined Vehicle Working Group of the Eclipse Foundation,” says Dr. Dirk Walliser, senior vice president Corporate R&D of the ZF Group. “The first projects presented and the growing number of members show that we are on the right track to improve the quality of automotive software through open source development. We see all vehicles as software-defined, think ‘software-first’ and act in a networked way.”

About the Eclipse Foundation

The Eclipse Foundation provides our global community of individuals and organizations with a mature, scalable, and business-friendly environment for open source software collaboration and innovation. The Foundation is home to the Eclipse IDE, Jakarta EE, and over 400 open source projects, including runtimes, tools, and frameworks for cloud and edge applications, IoT, AI, automotive, systems engineering, distributed ledger technologies, open processor designs, and many others. The Eclipse Foundation is an international non-profit association supported by over 330 members, including industry leaders who value open source as a key enabler for their business strategies. To learn more, follow us on Twitter @EclipseFdn, LinkedIn or visit eclipse.org.

Third-party trademarks mentioned are the property of their respective owners.

Media Contacts

Schwartz Public Relations for the Eclipse Foundation, AISBL

Stephanie Brüls / Susanne Pawlik

Sendlinger Straße 42A

80331 Munich

EclipseFoundation@schwartzpr.de

+49 (89) 211 871 – 64 / -35

Nichols Communications for the Eclipse Foundation, AISBL

Jay Nichols

jay@nicholscomm.com

+1 408-772-1551

From all of us at the Eclipse Foundation, we’d like to wish the LGBTQ+ community a happy Pride Month.

As June winds down, we’d like to recognize the Eclipse Community’s continuing efforts to foster an open and welcoming environment for everyone. 

Our Community Code of Conduct outlines the standards of behavior for everyone in our community. These standards include the usage of inclusive and welcoming language, being respectful of differing viewpoints, and showing empathy towards members of differing communities.

With contributors and committers from all over the world with different life experiences, we understand that it takes all of us to maintain a culture where everyone can feel like they belong. We also understand that our efforts to create safe and accepting spaces for people of all races, faiths, sexual orientations and gender identities do not end at simply upholding our Code of Conduct. 

That’s why we’re welcoming feedback on how we can increase our inclusion and diversity efforts. If you have any ideas on how we can work to improve in this area, please contact  emo@eclipse.org.

Are you interested in moving tools and IDEs into the cloud? Do you want to learn about challenges, best practices and...

The post Moving Tools to the Cloud: Challenges and Best Practices appeared first on EclipseSource.

Open source software is the single most important engine for innovation today. The ability to freely combine software components, frameworks, and platforms frees developers from constantly reinventing the wheel and allows them to focus on the new innovations that users want. Free software also enables business models to scale in ways that proprietary software would never allow. Globally and in all sectors of the economy, building on top of open source software is the dominant approach to delivering successful software systems today. 

However, with great success comes great responsibility. From Heartbleed to SolarWinds to Log4j, securing open source software and its global supply chain has never been more important. The reasons for this are many, but among them is that for too long open source has been treated by many of its consumers as “free as in free beer” where they should have been treating it as “free as in a free puppy.” Contributing to the sustainability of the projects and communities that deliver open source is really no longer a choice. It is a necessity.

At the Eclipse Foundation, we believe that foundations have a role to play in addressing the challenges of securing open source and its supply chain. Specifically, we want to provide services to our projects that help improve their security posture. But doing so requires additional staff and resources. That’s why we are so grateful for the financial support from the OpenSSF’s Alpha-Omega project, being announced today. This money will allow us to start building a team to roll out many of the ideas in our Open Source Software Supply Chain Best Practices document under the leadership of Mikael Barbero, our Head of Security. 

Some of the ways that we are going to put this funding to good use include:

• Automate the generation of static source-based SBOMs for all Eclipse Foundation project repositories.
• Implement a SLSA-based project badging program for Eclipse Foundation projects.
• Initiate a number of security audits for high-profile Eclipse Foundation projects.

We are also going to provide regular and public updates to the community about our progress and initiatives.

Software security is a never-ending process. This funding is the first step in a journey. We appreciate the support of the Alpha-Omega project, and are committed to using it effectively. 

More news

If you update to Xtext 2.27.0 and have compiler tests for your Xbase DSL that assert the output of the compilation, you’ll get lots of failures after the update.

I am guilty of that
Well, for a good reason, at least

In fact, I worked on this issue: https://github.com/eclipse/xtext-extras/issues/772 and its fix is included in Xtext 2.27.0.

Now, the Xbase compilation mechanism does not generate useless empty lines anymore (before, it added lines with two spaces). Your compiler tests will fail because the output is different.

I personally fixed my tests in my DSLs by simply using the Find/Replace mechanism of Eclipse with this substitution pattern (there are two space characters between the tab character and the newline character):

If you have deep nesting in your compilation output, you might have to repeat this substitution with more than two characters, but this should not be required unless you generate nested classes or something like that.

With the above substitution a test like the following one:

will become like the following one (you see the difference: no empty line with two characters between the two generated constructors:

Now your tests should be fixed

Do you want to develop a customized tool for C/C++ development in the web/cloud? Do you wonder which open source components...

The post Building Custom C/C++ Tools in the Web appeared first on EclipseSource.

We are happy to announce the Eclipse Theia 1.26 release! This release contains 50 merged pull requests and we welcome ten...

The post Eclipse Theia 1.26 Release: News and Noteworthy appeared first on EclipseSource.

A story of 40 years of coding and still feeling like a kid in a toy store.

My programming story begins

I love coding. I started coding in BASIC when I was 11 years old. I am from a fortunate generation, which was the first to have access to personal computers in their homes. I first started teaching myself BASIC on an Epson HX-20. The Epson HX-20 is regarded by some as the world’s first laptop computer. I still regret having sold this wonderful piece of computing history when I was in college.

I spent hours reading the manuals that came with the computer, and wrote my first program which calculated weekly pay with overtime. I was soon experimenting with the graphics programming capability of the computer as well. This is when I discovered the Joy of Programming. I was hooked.

Telling my story

I’m now 51 years old, and after 40 years of coding in 20+ different programming languages, I am happy to say that I am still hooked.

For the past twenty years, I have been telling stories about my career and love of programming to kids between the ages of 15 and 100. I always stress that I will “keep coding until someone pries my cold dead fingers from the keyboard.” This may sound a bit morbid, but gets the point across clearly. Programming is my passion and I am fortunate to be paid to do what I love. Over the past 20 years, I realized that in addition to writing code myself, I love to teach others how to code.

Good stories have heroes

The best way to predict the future is to invent it.

-Alan Kay, 1971

Pictured above is an XO laptop I bought during the One Laptop Per Child Give One Get One program in 2007. The program running on the screen is Squeak Etoys. Alan Kay is credited as the designer of Etoys, which was built in Squeak Smalltalk.

I was fortunate enough to meet Alan Kay while I worked at Goldman Sachs. He was invited to be on a Talks at GS Panel while I was an employee. I love that I can watch this talk and relive this experience at any time. If you don’t know who Alan Kay is, or just haven’t seen this talk before, you can see and hear Alan tell some of his story in link above. Alan Kay was kind enough to sign my XO laptop when I met him the same day as this talk— February 26, 2015.

Good stories often have heroes. Alan Kay is one of my computing heroes. I had no idea who Alan Kay was when I was 11 years old. I didn’t know who Alan Kay was until I was in my mid-twenties. For some reason, I was destined to learn about, be forever changed by and eventually meet my hero.

The 1980s

During the mid to late-80s, I learned a bunch of programming languages including BASIC, FORTRAN, COBOL, Pascal, Logo, Prolog, Dbase III+ and Clipper ‘87. I taught myself Dbase III+ in the mid-1980s, and then landed a job right after high school as a part-time consultant working with Clipper Summer ’87. This helped me pay for my university education at Rutgers Newark/NJIT.

The 1990s

After I finished school, I got a job working as a corporate programmer in the Rating and Underwriting department of Blue Cross Blue Shield of New Jersey (BCBSNJ). I start out programming in Clipper 5.x, and later would learn and program in the Windows version of Clipper which was called CA-Visual Objects. I spent all of my early career working in DOS, Windows 3.1, OS/2 and then Windows 95/NT.

In the mid-90s, BCBSNJ outsourced its IT department to Integrated System Solutions Corporation (ISSC) which later became IBM Global Services. I became an IBM employee overnight. At first, this was a bit scary, but it turned out to be a great opportunity in my career. Working at IBM would lead me down the path of learning the single most important programming language in my career. Along the way, that’s when I would I would discover who Alan Kay is and the impact he has had on modern computing.

In October 1994, I attended a 5 week hands-on immersion program in Object-Oriented Programming with Smalltalk. The program was part one of IBM’s Object Technology University. I spent 5 weeks in Atlanta, Georgia learning OO programming and design with IBM’s VisualAge Smalltalk. The classes lasted 6–8 hours each day, and we were taught by amazing industry experts in OO design and Smalltalk like Ghica van Emde Boas. I spent the other 4 to 6 hours left in the day coding and exploring VisualAge Smalltalk in the lab. I also read my first book on Object-Oriented Design written by Rebecca Wirfs-Brock during the 5 week program. Rebecca Wirfs-Brock is another one of my programming heroes. I consider myself fortunate to have signed copies of both of her OO design books.

Smalltalk cemented my love of programming. It is a beautiful language and immersive learning environment. I needed to share this new found knowledge with the world.

When I returned to work, I endeavored to find a project that I could build using IBM VisualAge Smalltalk. In the mean time, I continued building applications using CA-Visual Objects. I was now able to apply my new OO programming and design skills in this Object-Oriented language. I was energized and I wanted to share that energy with others. So I recommended sending more developers I worked with to the 5 week Smalltalk immersion program. After a year or so, I started working on a project building a Medicare Enrollment system using IBM VisualAge Smalltalk with two other developers who attended the Smalltalk immersion program. We brought in expert mentors from IBM’s Rapid Solutions Group to work with us. We leveraged and learned a persistence framework that they built on the project. It was a great learning experience.

I then went on to re-design and re-implement all of the Corporate Rating and Underwriting Systems I had previously written in Clipper, only now using IBM VisualAge Smalltalk. During this effort, which lasted about three years, we hired an expert consultant in Distributed Smalltalk. He was a mentor for myself and another colleague, and he taught us a lot about how to think about building and testing distributed systems. He also taught us about the history of programming and Smalltalk. This is when I learned who Alan Kay and Dan Ingalls were from him. Dan Ingalls is another one of my programming heroes, who wrote an amazing paper titled the “Design Principles Behind Smalltalk.”

Y2K+

In the year 2000, I decided to leave my beautiful coding life as a Smalltalk developer, to go code in Visual Basic and more Enterprise-popular languages like Java, because that is where the industry was focusing. My former Distributed Smalltalk mentor told me to go learn Java and become an expert in it, so I did. I was happy about the possibility of the future, but miserable that I had to code in Visual Basic and Java. I had traded in a flying DeLorean time machine for a Schwinn Bicycle and a Ford Truck. The bicycle looked new and shiny, but was mostly useless for anything but trivial apps. The truck could do a lot of heavy lifting but required a lot of gas (developer effort).

I joined Goldman Sachs (GS) in 2001. I started off as an application architect, was promoted to VP, spent a year living in London, became the CTO for Controllers Technology, was promoted to Tech Fellow, built an amazingly talented Core Platform Team, created an evolving presence for GS on GitHub, and was promoted to Managing Director. That’s about thirteen years in a sentence. I programmed for 15+ years in Java while at GS, and created a collections library in Java that was open sourced as GS Collections in 2012 and eventually became Eclipse Collections by the end of 2015.

Why did I create Eclipse Collections? After five years of programming in Java, I got tired of repeating the same coding patterns over and over again. I recognized much of what was missing in Java, because I had seen so many advanced features while programming in Smalltalk. I found it challenging to explain to developers at the time what they were missing in Java. So I decided to build a few of the features I was missing from Smalltalk in Java. I have taught many Java developers about Smalltalk, OOP, OOAD, Lambdas, TDD, Refactoring over the years. These were all things I had learned in Smalltalk in the 1990s.

The Joy of Programming

The most important lesson I have taught developers over the years is that Programming is Fun. If you are programming, and you are not having fun, then you may be doing it wrong. It may not be your fault. Ask for help. We are at our most creative when programming is fun. Creativity is play. We need to be creative to solve the complex problems we are faced with today.

I am very passionate about Eclipse Collections, but it’s not because I was ever particularly interested in collection frameworks. It’s because by using Eclipse Collections I can teach a Java developer how to have real fun while programming in the Java programming language. I have met so many Java developers over the years who either became managers or quit programming altogether because the language seemed to beat the love of programming out of them.

It does not have to be that way. I hated programming in Java for the first few years, but I never lost my love of programming. I certainly missed programming in Smalltalk and Clipper because they were so much more fun and productive to work with. I spent several years working on the Java Specification Request (JSR) 335 Expert Group (EG) with industry experts like Brian Goetz, Doug Lea, Dan Heidinga, Rémi Forax, Sam Pullara, Tim Peierls, Bob Lee, Kevin Bourrillion, Andrey Breslav, Vlad Zakharov and a few others. We all wanted to improve the Java language by introducing lambdas to make it more fun and productive to program in. I can look back now and say I helped the JSR 335 EG “invent the future” for Java. Now Java programming is way more fun, and much less pedestrian to work in since Java 8. But there are still so many developers out there who have yet to experience the joy. Help is available!

I still love Smalltalk, and I’m sure I always will. I have now learned to love programming in Java as well. I could have thrown in the towel and chosen a different language to program in, or a move to a completely different profession. I believed programming is fun, and I worked hard with talented experts in the community to help make Java more fun, not just for myself, but for millions of other Java developers.

Sharing is Caring

Do what you love. Love what you do. Get paid for it if and while you can. Pay it forward. Share your joy of programming with others. Motivate and inspire future generations to achieve all they are capable of. Then perhaps you might understand and appreciate the Joy of Programming as I do.

I gave a talk recently at the Pittsburgh Java User Group titled “Smalltalk or Java? Why not Both!”. There is a YouTube recording linked from the following blog. You can see me live coding in a Smalltalk IDE, which is something I haven’t done in a very long time. This is a great example of me sharing my Joy of Programming in Smalltalk and Java with others. If you take the time to watch it, thank you! I hope you enjoy it!

Smalltalk or Java? Why not both!

The Future

After 40 years of coding, I have decided to write a lot more frequently. I am on a personal mission to share as much as I can remember about Java, Eclipse Collections and Smalltalk. I am writing 1–2 blogs a month on Medium now. I will have been blogging publicly for the past five years by August of this year. I want to do my part to make the future better, and to keep the joy of programing alive for future generations.

I was given a wonderful gift seven years ago from one of my computing heroes — Alan Kay. The XO laptop he signed for me is something I will always cherish. Alan couldn’t know at the time, as he had just met me, but what he wrote on the laptop felt like a prophesy to me. He wrote “The future will be better. ” At the time he wrote it, my wife was preparing to enter the hospital a month later to have a stem cell transplant in her battle against AML. I needed to believe what he wrote. This signed laptop literally brought me to tears when I brought it home and shared the story with my wife. Alan had shared the gift of optimism.

Everything at any current time in your work or personal life may completely suck. Current events would seem to prove this the case for many, but the optimist in me believes what Alan wrote is true. The future will be better, but only if we work very hard to make it that way. If we work together, collaborate and invest in each other, we can invent a better future.

One of the joys I have learned about programming in Java that I hadn’t gotten to experience in Smalltalk is the joy of working with a large, collaborative and supportive community. The Java community is absolutely incredible. It is the gift that keeps on giving. I am humbled and honored to be a part of this vibrant community. Together, we will continue to make the future better for the entire Java community. The Joy of Programming continues.

If you’d like to read a detailed version of my Java story from the past two decades, I wrote about it when Java celebrated its 25th anniversary.

Happy 25th Birthday Java

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

This new program gives software and hardware providers the means to showcase compatibility and certify products with the Eclipse Sparkplug protocol and MQTT-based IoT infrastructure

ORLANDO, Fla., JUNE 7, 2022 – The Eclipse Foundation, one of the world’s largest open-source software foundations, in collaboration with its Sparkplug Working Group, today announced at the ARC Industry Forum the launch of the Sparkplug Compatibility Program. This program is based on Sparkplug, an open source software specification that enables mission-critical operational technology (“OT”) clients to use industry standards like MQTT to seamlessly integrate data from their applications, sensors, devices, and gateways with most Industrial Internet Of Things (IIoT) Infrastructure.

“MQTT has already established itself as the ‘de facto’ standard for messaging transport in the IT and OT market sectors,” said Mike Milinkovich, executive director for the Eclipse Foundation. “However, it doesn’t specify the content of its payloads, making interoperability across the IIoT incredibly challenging. Sparkplug, acting as the HTML of the IIoT, is the industry’s best solution for solving this issue and is already in widespread use across multiple industries. Our new program lets industrial firms know if their vendors’ systems are Sparkplug compatible.” 

The Sparkplug Compatibility Program aims to provide integrators and end-users with an easy way to procure devices and software products that are fully compatible with the Sparkplug specification, thus ensuring their solutions will seamlessly integrate with the most common Industrial Internet of Things (IIoT) devices and networks. To be included in the program, products will need to pass an open-source series of tests that validate conformance to the specification. These tests are based on the Sparkplug Technology Compatibility Kit (TCK). Products passing the TCK will be featured in the official list of compatible products, available on the Sparkplug Working Group’s website. In addition, licensees of the Sparkplug Compatible trademark will be able to promote compatibility, while also being recognizable in the marketplace through the use of the “Sparkplug Compatible” logo. 

Organizations who wish to participate in the compatibility program are invited to join the Eclipse Foundation and Sparkplug Working Group. Interested parties can find out more about the Sparkplug Working Group and the Sparkplug Compatibility Program here.

About Sparkplug & MQTT

By design, the MQTT specification does not dictate a Topic Namespace or any payload encoding. However, as the IIoT and other architectures leveraging the publisher/subscriber model are adopted by device OEMs in the industrial sector, having different Topic Namespace and payload encoding can inhibit interoperability for the end customer.

Sparkplug provides an open and freely available specification for how Edge of Network (EoN) gateways or native MQTT enabled end devices and MQTT Applications communicate bi-directionally within an MQTT Infrastructure. It is recognized that MQTT is used across a wide spectrum of application solution use cases, and an almost indefinable variation of network topologies. To that end the Sparkplug specification addresses the following components within an MQTT infrastructure: 

• Sparkplug defines an OT-centric Topic Namespace 
• Sparkplug defines an OT-centric Payload definition optimized for industrial process variables 
• Sparkplug defines MQTT Session State management required by real-time OT SCADA systems

Quotes from Participating Organizations 

Cirrus Link Solutions

“Cirrus Link was founded with the express purpose of providing MQTT centric software for Industrial Automation Solutions. As one of the co-inventors of MQTT I couldn’t be more pleased to be participating with the Eclipse Sparkplug Working group and see the launch of the Sparkplug Compatibility program. Having native devices and software solutions providing “Plug and Play” capabilities leveraging the power of an MQTT/Sparkplug infrastructure will be a disruptive change in the way industrial automation solutions are designed and implemented in the future.” Arlen Nipper, CTO at Cirrus Link Solutions

Chevron

"For a large enterprise like Chevron, Automation Engineers around the globe could be spending countless hours testing to see if something will work in their process control network or their IIoT network. Having the confidence that something will "just work" when you plug it in to your system via the Sparkplug compatibility program is a huge saver of time and money." Todd Anslinger, IIoT & Automation Specialist at Chevron.

Inductive Automation

“We are definitely excited about the Sparkplug Compatibility Program and the experience of collaborating with this Eclipse Working Group. Since day one, Inductive Automation has supported an open, interoperable, and standards-based approach to industrial automation systems. The Sparkplug Specification expands Digital transformation opportunities across the industrial sector enabling deployments at scale and accelerating time to value. The entire ecosystem supporting IIoT evolution benefits from the launch of the Sparkplug Compatibility Program.” Don Pearson, Chief Strategy Officer, Inductive Automation

Opto 22

“For our customers, democratizing data in OT systems while securing legacy, brownfield devices is top of mind,” states Benson Hougland, VP Product Strategy with Opto 22. “MQTT with Sparkplug provides the tools they need to safely and simply share data among industrial operations, IT systems, and cloud platforms. Back in 1996, Opto 22 joined with Microsoft to develop the OPC specification—and later found the OPC Foundation—to solve the data share challenges of that decade. Today’s data-sharing problems have evolved. Our Sparkplug Working Group membership is our commitment to offer the most effective solutions now and ensure our customers’ success.”

HiveMQ

“Sparkplug is changing the OT industry for the better,” states Ian Skerret, VP of Marketing at HiveMQ. “It solves the data interoperability challenges many of our customers have rolling out new IIoT systems. HiveMQ is proud to participate in the collaborative community to make Sparkplug a success.”

Canary

“Our time-to-value when working with enterprise applications using Sparkplug brokers is mind blowing. Whether 10,000 tags or 2 million tags, historizing Sparkplug tags into a Canary Historian happens instantaneously,” said Jeff Knepper, executive director, Business Development, Canary.

About the Eclipse Foundation

The Eclipse Foundation provides our global community of individuals and organizations with a mature, scalable, and business-friendly environment for open source software collaboration and innovation. The Foundation is home to the Eclipse IDE, Jakarta EE, and over 400 open source projects, including runtimes, tools, and frameworks for cloud and edge applications, IoT, AI, automotive, systems engineering, distributed ledger technologies, open processor designs, and many others. The Eclipse Foundation is an international non-profit association supported by over 330 members, including industry leaders who value open source as a key enabler for their business strategies. To learn more, follow us on Twitter @EclipseFdn, LinkedIn or visit eclipse.org.

Third-party trademarks mentioned are the property of their respective owners.

###

Media contacts:

Schwartz Public Relations for the Eclipse Foundation, AISBL

Stephanie Brüls / Susanne Pawlik

Sendlinger Straße 42A

80331 Munich

EclipseFoundation@schwartzpr.de

+49 (89) 211 871 – 64 / -35

Nichols Communications for the Eclipse Foundation, AISBL

Jay Nichols

jay@nicholscomm.com

+1 408-772-1551

Learn the forEach and injectInto patterns in Eclipse Collections.

To forEach or to injectInto?

In Eclipse Collections, forEach and injectInto are both internal iterators that provide the most basic of the iteration patterns. A developer can use both of these patterns to accomplish a large number of iteration tasks. As internal iterators, these methods encapsulate the implementation details of “how to” iterate over the elements of the collection, and leave it to the developer to use a lambda to specify the behavior that should be applied to each element of a collection.

The forEach and injectInto patterns can be used as the building blocks for many of the other iteration patterns (e.g. select, reject, collect, etc.). Both patterns iterate over a collection from the beginning of the collection to the end and perform some operation that receives each element of the collection as a parameter. Neither the forEach nor the injectInto pattern can be used effectively to implement any of the short-circuiting patterns (e.g. detect, anySatisfy, allSatisfy, noneSatisfy).

The functionality of forEach and injectInto can be summarized as follows.

• forEach takes a single argument Procedure and returns void. Each elements of the collection is passed to the Procedure.
• injectInto takes a two argument Function2. A value is injected in as the first parameter of the Function2 along with each element of the collection. The method returns the final result that the Function2 returns after the last element is processed.

There are specializations of both patterns. The injectInto pattern has more primitive specializations than forEach because injectInto returns some type and it may return one of the eight primitive types, or Object. The forEach patterns always return void.

One of the things that injectInto can do, that forEach cannot, is perform a side effect free operation. With forEach, a side effect always happens somewhere. That is its purpose — to make side effects happen with elements of the collection. There may be a mutation applied to an element of the collection, or a mutation to a variable in the scope of a lambda, or the element is printed to System.out, etc.

With injectInto, it is possible to perform a side effect free operation because it injects, returns, and re-injects some result from the two-argument Function. The value that is injected and returned may be immutable, like a String or Integer or a primitive value.

I’m going to show some examples of using forEach and injectInto to solve similar problems. This should begin to make it clear when one of the patterns is more suitable or desirable than the other. The answer of whether one is more desirable than another will be an exercise left up to you, the reader. In some cases, forEach will be more readable. In many cases, injectInto will be more flexible, safe and/or performant.

The following are the examples I will show:

• Count
• Sum
• Convert to Set

Example: Count

In the following examples, I will count the even integers contained in a MutableList<Integer> and MutableIntList. There is a method named count defined on both Object and Primitive collections in Eclipse Collections that takes a Predicate as a parameter. The intent of these examples is to show some options for implementing count using forEach or injectInto patterns.

1. Count: forEach (Object List)

I had to create a mutable Counter in order to record the total count of even values. I used the ternary operator to write a simplified if expression to determine whether to add a value of 1 or 0 to the Counter.

2. Count: forEach (Primitive List)

The only difference between the forEach on the object and primitive lists is that the int values in the primitive List don’t require unboxing as the Integer values do in the object List. This approach still requires having an effectively final mutable Counter object to increment in the lambda.

3. Count: injectInto (Object List)

This implementation is side-effect free. We start with an initial Integer value of 0, and add a 1 or 0 to that result for each element if it is even. There is a lot of unboxing and auto-boxing here as the result of each function will unbox values and then auto-box the result.

4. Count: injectInto (Primitive List)

This example uses a MutableIntList which does not box the int values in the List. The result of injectInto is still an object, so the results are auto-boxed for each element.

5. Count: injectIntoInt (Object List)

The specialization injectIntoInt allows an int value of 0 to be injected into the iteration and returned as the result. There will be no auto-boxing of the results here, but the values are unboxed in order to test if they are even. This solution is both side-effect free and results in zero auto-boxing. The values in the integers List however are still boxed as it is a MutableList<Integer>.

6. Count: injectIntoInt (Primitive List)

The specialization injectIntoInt on a MutableIntList allows for the implementation of side-effect free counting with no boxing of int values to Integer objects in either the list or the calculations.

Example: Sum

In the following examples, I will sum the integers contained in a MutableList<Integer> and MutableIntList. There is a method named sumOfInt defined on Object collections and simply sum on Primitive collections in Eclipse Collections. The intent of these examples is to show some options for implementing sum using forEach or injectInto patterns.

1. Sum: forEach (Object List)

I used LongAdder from the JDK to accumulate the sum. There is an add method on LongAdder that takes a long. Each Integer object in the integers List is unboxed and cast to a long.

2. Sum: forEach (PrimitiveList)

The primitive version of sum with forEach is essentially the same as the object version, just without requiring the unboxing of Integer objects in the call to add on LongAdder.

3. Sum: injectInto (Object List)

By injecting an initial value of Long.valueOf(0), the return type of injectInto will be a Long. I used the Long::sum method reference which takes two long values as parameters. The result is unboxed from Long to long and the Integer values in the List are unboxed and cast to long.

4. Sum: injectInto (Primitive List)

The primitive version of sum using injectInto using Long as an injected value is essentially the same as the object version. The one difference is there is no unboxing of Integer objects required.

5. Sum: injectIntoLong (Object List)

Using injectIntoLong on an object List and injecting an initial long value of 0L, the return type of injectIntoLong is a primitive long. I used the Long::sum method reference which takes two long values as parameters. The result is passed in as a long and the Integer values in the List are unboxed and cast to long.

6. Sum: injectIntoLong (Primitive List)

Using injectIntoLong on a primitive List and injecting an initial long value of 0L, the return type of injectIntoLong is a primitive long. I used the Long::sum method reference which takes two long values as parameters. The result is passed in as a long and the int values in the primitive List are cast to long.

Example: Convert to Set

In the following examples, I will add the integers contained in a MutableList<Integer> or MutableIntList into a MutableSet<Integer> or MutableIntSet. I will also show how injectInto can be used to add to an ImmutableSet<Integer> or ImmutableIntSet.

There are converter methods on Object and Primitive collections for converting from one type of container to another (e.g. toSet, toImmutableSet). The intent of these examples is to show some options for implementing toSet and toImmutableSet using the forEach or injectInto patterns.

1. To Mutable Set: forEach (Object List)

I create a MutableSet<Integer> named set to hold the elements I want to transfer from integers. Then using forEach, I pass the method reference set::add. This code is really simple and straightforward.

2. To Mutable Int Set: forEach (Primitive List)

The primitive version is essentially same as the object version, with the target set being MutableIntSet.

3. To MutableSet: injectInto (Object List)

Using injectInto, I inject the MutableSet<Integer> as the first parameter of the Function2. I use the MutableSet::with method reference which matches the two-parameter types required by the Function2.

The first parameter type isMutableSet<Integer> and the second parameter type is the type of each element of the collection (Integer).

The return type of with on any MutableCollection in Eclipse Collections is the collection type itself, since with calls add and then returns this.

4. To MutableIntSet: injectInto (Primitive List)

The primitive version is essentially same as the object version, with the target set being MutableIntSet.

5. To ImmutableSet: injectInto (Object List)

Converting a MutableList to an ImmutableSet id another example that demonstrates the flexibility of injectInto over forEach. An ImmutableSet has no mutating methods. There is no add method like a MutableSet. There is a newWith method which creates a new ImmutableSet by copying the original set and adding an element to create a new ImmutableSet. I pass an empty ImmutableSet into injectInto along with a method reference to ImmutableSet::newWith.

6. To Immutable Int Set: injectInto (Primitive List)

The primitive version is essentially same as the object version, with the target set being MutableIntSet.

Additional examples of forEach

Sometimes you might need an index with your forEach. Thankfully, in Eclipse Collections, there is forEachWithIndex for this purpose. Here’s a blog that describes methods that provide indexes including forEachWithIndex.

I need an index with this List iteration method

There is also a blog that describes a specialized form of forEach named forEachInBoth. Since this blog was originally written, I have added forEachInBoth directly to Object and primitive List containers in Eclipse Collections.

Mysterious Eclipse Collections APIs: forEachInBoth

Additional examples of injectInto

There are a couple of blogs that cover more examples of injectInto as well as specialized forms of injectInto like injectIntoKeyValue. The following blog covers some general examples of injectInto.

EC by Example: InjectInto

The following blog describes and shows examples of injectIntoKeyValue which works with Maps and primitive Maps in Eclipse Collections. The blog was written by Emilie Robichaud.

InjectIntoKeyValue in Eclipse Collections

Some final thoughts on forEach and injectInto

I find it is very easy to start thinking about a problem using forEach. If you need to cause some basic side effect, forEach is a very simple go to method.

Sometimes after I have written something using forEach, I wonder what the same code would look like if I used injectInto. I will use injectInto if I want to have a side effect free operation, or to control any side effects fully within the context of the Function2 that I pass to injectInto.

So should you use forEach or injectInto in Eclipse Collections? As with many things, the best solution may be subjective, and I would recommend going with the code that you find the easiest to read and understand.

If you do learn how to add injectInto to your bag of tricks, you may start to be amazed at its power and flexibility, even if its mystery never quite goes away.

Source for Examples

Want to Connect?

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Should You Use ForEach or InjectInto in Eclipse Collections? was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.

The launch of the Adoptium Marketplace on May 26 is exciting news for the millions of developers, researchers, and organizations who rely on TCK-tested compatible Java runtimes. As noted in the announcement, by providing a vendor neutral home for the OpenJDK ecosystem, the marketplace makes it easier than ever to access Java SE-conformant binaries necessary for cloud native and enterprise deployments.

But there’s more to the story. For a long time, compatibility has been the name of the game when it came to Java implementations. The Adoptium Marketplace has been set up to take the Java ecosystem to the next stage of its development. 

That’s where Eclipse AQAvit comes in. It brings quality assurance metrics into the marketplace, so that the Java community can begin to select binaries not just based on compatibility but on quality. 

Eclipse AQAvit Brings Quality Assurance to Java

Everything in the marketplace will be compatible with the relevant version of the Java SE Technology Compatibility Kit (TCK). 

But TCK compatibility doesn’t tell you anything about the quality of the implementation. In recent years, the number of OpenJDK-based runtime distributions has absolutely exploded. And although many vendors maintain their own release quality tests, OpenJDK distros have historically not been built to any consistent quality standard. It has become increasingly clear that the Java ecosystem needs a consistent, multi-vendor definition of quality.

Ensuring high-quality binaries are ready for production deployment is crucial for the Adoptium Marketplace. The AQAvit project team compiled tens of thousands of tests and built a few of their own to produce a comprehensive, systematic way of ensuring the quality of runtimes available. The AQAvit Quality Verification Suite covers a broad set of requirements, ensuring binaries provide superior: 

• Performance
• Security
• Resilience
• Endurance

They also ensure that the binaries can pass a wide variety of application test suites and can verify new functionality during runtime development. That’s what’s unique about the Adoptium Marketplace: it provides peace of mind knowing that the binaries are not only compatible but will actually meet the demanding requirements of your enterprise applications.

Contributing Helps Ensure AQAvit Meets Your Needs

And in the spirit of open source, you give a little to get a lot.

Many of the founding members of the Adoptium Working Group are Java developers and vendors, including Alibaba Cloud, Azul, Huawei, IBM, iJUG, Karakun AG, Microsoft, New Relic, and Red Hat. The marketplace enables working group members to promote their Java SE compatible releases verified to Eclipse AQAvit’s quality criteria. Their membership helps support the cloud-based infrastructure that drives Adoptium’s efficiency as a shared community project. In other words, the working group collaborates to create and provide access to high-performance, enterprise-caliber, cross-platform, open source-licensed, and Java-compatible binaries of OpenJDK builds, through the marketplace. 

Contributing to the AQAvit project is one of the best ways to ensure access to runtimes that meet specific needs. We encourage Java community members to get involved and contribute additional tests to cover the use cases their applications require. They’ll be incorporated in the AQAvit test suite, so every binary going forward will have to meet that standard. This way enterprises and developers can be confident any AQAvit-verified binaries they use will function as needed. 

Security Updates for Java

Quality assurance is a big part of what makes the Adoptium Marketplace unique, but it’s not the whole picture. Security fixes are also an important focus.

Once upon a time, you could count on getting security fixes for old versions of Java for a long time. After all, if you’ve deployed a set of applications on a version, you’re probably going to want to use it for a long time. 

That’s no longer the case elsewhere. But all the distributions in the Adoptium Marketplace will be kept up to date with the latest security patches or those patches will be backported to older LTS versions. This way you can be sure that your applications are secure, no matter which version of Java you’re running them on. Of course, this goes for new versions of Java too.

Everything Users Need in One Place

The Adoptium Marketplace brings together all these elements — quality assurance, adaptability to community needs, security updates for every version, sustainability — into a one-stop shop for binaries. Ultimately, this delivers five key assurances to end users:

• The binary has been tested and is compatible with the relevant version of the Java SE TCK
• The binary was built in accordance with open source principles
• The binary has been fully verified using the AQAvit quality verification criteria, having passed through multiple tests to ensure it meets industry quality standards
• The binary is as secure as possible, with the latest security updates included
• The binary is brought to you by a vendor committed to supporting and participating in a multi-vendor, vendor-neutral collaboration

If your organization is considering participating in the Adoptium Working Group, have a look at the Charter and Participation Agreement. Or if you have questions, email us at membership@eclipse.org. 

Are you interested in building a custom IDE or tool for your domain that runs in the cloud but also as...

The post Getting started with Eclipse Theia appeared first on EclipseSource.

Submit your papers for Eclipse SAAM on Cloud, a conference that will bring together practitioners and researchers working on innovative software and systems solutions for next-generation mobility.

This year the conference is co-located with EclipseCon. Co-organized by the Eclipse Foundation, ATB-Bremen and CERTH, the conference will focus on Security and Privacy, Artificial Intelligence, Architecture, Modelling and related challenges.

Read the Call for Papers.

I love lambdas in Java 8, but method references are elusive and amazing

Lambdas are flexible anonymous bits of code

We can solve a lot of interesting problems in Java using lambdas. We can use expression lambdas for simple things, and statement lambdas for more complex things. Lambdas can call out to other methods on the current object (this) or objects that are in scope, like the current element of an iteration, or a final local variable outside of the lambda. We can always simplify a lambda by putting code in another method.

Writing good lambdas requires discipline. For instance, it is important to use intention revealing names for parameters. Here’s a simple example of a using a lambda to filter a List of Strings.

@Test
public void filterStringsLambda()
{
    var list = Lists.mutable.with("Atlanta", "Atlantic City", "Boston", "Boca Raton");

    var actual = list.stream()
            .filter(string -> string.startsWith("At"))
            .collect(Collectors.toList());

    var expected = List.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected, actual);
}

In this code, the lambda is the parameter passed to the filter method in the form of a Predicate. In this example, the Predicate takes a parameter of type String which I give the name string. The expression after the separator (->) will be evaluated for each element of the list and will only include those elements which evaluate to true.

There are several methods in the Stream API that will take a Predicate as a parameter. The methods include filter, anyMatch, allMatch, noneMatch.

There is no easy way for me to use a method reference here because I need to pass the parameter “At” to the method startsWith. Parameters are kryptonite for method reference usage. We can simulate a method reference here by using a lambda and extracting it into a separate method as follows.

@Test
public void filterStringsLambdaInMethod()
{
    var list = Lists.mutable.with("Atlanta", "Atlantic City", "Boston", "Boca Raton");

    var actual = list.stream()
            .filter(this.stringStartsWith("At"))
            .collect(Collectors.toList());

    var expected = List.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected, actual);
}

private Predicate<String> stringStartsWith(String prefix)
{
    return string -> string.startsWith(prefix);
}

Having to create a method on a class to generate lambdas that can leverage local variables in scope is less than ideal. I would love to be able to just use the startsWith method as a method reference.

How to satisfy a Method Reference Preference?

Use the With methods in Eclipse Collections.

This is the way

For many of the methods available in the Eclipse Collections API, there is a corresponding method with the suffix of With. Each With method takes a different named functional interface that takes two parameters (e.g. Predicate2, Function2, etc.). The following mind map shows some of the basic methods in the Eclipse Collections API along with their corresponding With equivalents and functional interface types they take as parameters.

How do these extra methods help you with using method references with parameters? Let’s walk through some examples.

Basic Using Lambda

Let’s see the example of filtering a List of Strings using one of the basic Eclipse Collections methods with a lambda.

@Test
public void selectStringsLambda()
{
    var list = Lists.mutable.with("Atlanta", "Atlantic City", "Boston", "Boca Raton");

    var actual = list.select(string -> string.startsWith("At"));

    var expected = List.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected, actual);
}

With Method Reference

Now let’s see how we can satisfy our method reference preference using the “With” equivalent of select.

@Test
public void selectStringsWithMethodReference()
{
    var list = Lists.mutable.with("Atlanta", "Atlantic City", "Boston", "Boca Raton");

    var actual = list.selectWith(String::startsWith, "At");

    var expected = List.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected, actual);
}

I have spoken

If you didn’t just have an “aha!” moment, don’t be alarmed. We still can’t pass parameters to method references directly. There is no syntax in Java currently to support it. There is a trick that is happening here.

Let me try and explain how this works. The method selectWith takes two parameters. The first parameter is a Predicate2 , which as it turns out will match the signature of String::startsWith. To be more specific, the Predicate2<String, String> matches the signature of String::startsWith. The second parameter selectWith takes is any type of parameter, which in this case happens to be a String.

Here’s the exact signature of selectWith on RichIterable.

<P> RichIterable<T> selectWith(Predicate2<? super T, ? super P> predicate, P parameter);

I added the next section after I initially published the blog. A friend suggested a minor improvement would be to include a “how to” example so developers could see there is no magic and can begin leveraging for their own code. Thanks for the suggestion Rustam!

A selectWith pattern implementation example

There is a class in Eclipse Collections named IteratorIterate. It includes many of the basic eager iteration patterns in Eclipse Collections that allow the patterns to be used with any Iterable type in Java. I’m sharing this example because Iterator is a basic enough concept that most Java developers should be able to read and understand the code. The following shows the implementation of selectWith in IteratorIterate that is method reference friendly for method references with a single parameter.

public static <T, P, R extends Collection<T>> R selectWith(
        Iterator<T> iterator,
        Predicate2<? super T, ? super P> predicate,
        P injectedValue,
        R targetCollection)
{
    while (iterator.hasNext())
    {
        T item = iterator.next();
        if (predicate.accept(item, injectedValue))
        {
            targetCollection.add(item);
        }
    }
    return targetCollection;
}

This pattern can be used with any type that can create an Iterator.

Here’s an example using IteratorIterate.selectWith with a JDK Set.

@Test
public void selectWithOnIteratorIterate()
{
    Set<String> strings = Set.of("Atlanta", "Atlantic City", "Boston", "Boca Raton");
    
    HashSet<String> actual = IteratorIterate.selectWith(
            strings.iterator(),
            String::startsWith,
            "At",
            new HashSet<>());

    var expected = Set.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected, actual);
}

I hope this additional section was helpful.

More Method References Please!

Now that we know how to use a method reference with a With method, let me show you some more examples.

@Test
public void predicatesWithMethodReference()
{
    var list = Lists.mutable.with("Atlanta", "Atlantic City", "Boston", "Boca Raton");

    var selected1 = list.selectWith(String::startsWith, "At");
    var expected1 = List.of("Atlanta", "Atlantic City");
    Assertions.assertEquals(expected1, selected1);

    var rejected = list.rejectWith(String::startsWith, "At");
    var expected2 = List.of("Boston", "Boca Raton");
    Assertions.assertEquals(expected2, rejected);

    var selected2 = list.selectWith(String::startsWith, "Bo");
    Assertions.assertEquals(expected2, selected2);

    var detected = list.detectWith(String::endsWith, "y");
    Assertions.assertEquals("Atlantic City", detected);

    var count = list.countWith(String::contains, "c");
    Assertions.assertEquals(2, count);

    Assertions.assertTrue(
        list.anySatisfyWith(String::contains, "a"));
    Assertions.assertTrue(
        list.allSatisfyWith(String::contains, "t"));
    Assertions.assertTrue(
         list.noneSatisfyWith(String::contains, "z"));

    var partitioned = list.partitionWith(String::endsWith, "n");
    Assertions.assertEquals(expected2, partitioned.getSelected());
    Assertions.assertEquals(expected1, partitioned.getRejected());
}

There are a lot of methods that take single parameters that can match Predicate2, Function2, Procedure2, etc. as method references. The With methods in Eclipse Collections increase the total number of places you can use method references instead of lambdas quite a bit.

Enjoy Method References and Lambdas

I hope this blog helped you discover a nifty feature available in Eclipse Collections that can help you find more places to use method references. We had the “With” methods in Eclipse Collections years before Method References arrived in Java 8. We initially added them so we could create more opportunities to hoist up anonymous inner classes into static variables to reduce garbage generation. We used to call these “fat free closures” as they didn’t require you to keep adding new objects to the heap. The coincidence that this made it easier to leverage these methods with method references was an amazingly pleasant and welcome surprise.

Thank you for reading this blog! I hope you get to enjoy using method references as much as I do now.

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Further Learning

• 10 Best Places to Learn Java Online for Free in 2022
• The 2022 Java Programmer RoadMap
• Top 22 Java Libraries and APIs Every Programmer Should Learn in 2022

The elusive and beautiful Java Method Reference was originally published in Javarevisited on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Adoptium Working Group gives millions of developers, academics, and researchers a single centralized source of fully verified Java binaries, including Eclipse Temurin

BRUSSELS, Belgium – MAY 26, 2022 – The Eclipse Foundation, one of the world’s largest open source software foundations, in collaboration with its Adoptium Working Group, today announced the Adoptium Marketplace. This new market gives users the opportunity to obtain Java SE TCK certified and AQAvit quality-tested binaries, including Eclipse Temurin, via a centralized site managed by the Eclipse Foundation. By doing so, the Eclipse Foundation and the Adoptium Working Group give millions of developers, researchers, and other organizations an easy means of securing high performance, enterprise-caliber, cross-platform, AQAvit verified, and TCK-certified binaries for general use across the Java ecosystem.

“With the increasing volume and diversity of OpenJDK runtime distributions, it’s become clear that quality and consistency across the ecosystem must be established, ” said Mike Milinkovich, executive director of the Eclipse Foundation. “The creation of the Adoptium Marketplace provides a vendor-neutral home for the Java ecosystem that makes it incredibly easy for anyone to find Java SE compatible binaries that have been extensively tested and are ready for cloud native and enterprise deployments.”

The Adoptium Working Group, which was founded by multiple participants, including many Java developers and vendors such as Alibaba Cloud, Azul, Huawei, IBM, iJUG, Karakun AG, Microsoft, New Relic, and Red Hat, provides the Java ecosystem with fully compatible, high-quality distributions of Java runtimes based on OpenJDK source code.

There has been an explosion in the number of OpenJDK-based runtime distributions available over the last few years, and not all of them have the high quality bar set by the Eclipse Foundation. The Adoptium project is hosting a website and application programming interface “marketplace” where working group members can promote Java SE runtimes that meet specific quality assurance criteria.

The Eclipse AQAvit project was created specifically as a means of testing the exacting criteria that will ensure a binary is ready for production deployment. This is a critical service, as thousands of enterprises and millions of developers that run Java need a reliable source of high-quality Java runtimes that come with timely patches and updates for their mission-critical Java applications. 

The Adoptium marketplace enables working group members to promote their own Java SE releases that meet Eclipse AQAvit’s quality criteria, and continue to innovate in the cloud-based infrastructure that drives our efficiency as a shared community project. The Adoptium Marketplace complements the Working Group’s efforts by delivering a trusted location where developers and enterprises can obtain compatible binaries of OpenJDK builds. Anyone choosing a binary from this Adoptium Marketplace can be assured of these primary benefits:

• The binary has been tested and passed the relevant version of the Java SE TCK to ensure Java compatibility;
• The binary has been fully verified using the AQAvit test criteria, having passed through multiple tests to ensure it meets industry quality standards; and
• The results of the AQAvit quality checks are available for public scrutiny. 

The Eclipse Adoptium project and governing Eclipse Adoptium Working Group are the continuation of the original AdoptOpenJDK mission, which was established in 2017 to address the general lack of an open, community-based, and reproducible build and test system for OpenJDK across multiple platforms. Eclipse Adoptium is now the leading provider of high-quality OpenJDK-based binaries used by Java developers in embedded systems, desktops, traditional servers, modern cloud platforms, and mainframes.  The Adoptium Marketplace extends this leadership role and gives even more organizations a means of distributing their binaries. 

If your organization is interested in participating in the Adoptium Working Group, you can view the Charter and Participation Agreement, or email us at membership@eclipse.org.

Quotes from Participating Organizations 

Temurin

“Eclipse Temurin by the Adoptium project is the leading open source distribution of enterprise-ready Java SE across multiple platforms,” said Tim Ellison, Adoptium PMC Lead. “Temurin is proud to participate in the Adoptium Marketplace alongside working group members who share our vision for highly scalable, secure, and performant runtimes. Through the marketplace, users are assured that the Java they choose for their applications meets the demanding quality verification offered by AQAvit tests.”

Azul

“Market adoption of OpenJDK runtimes is rising fast as users explore their options for moving off costly legacy Java vendors,” said Simon Ritter, Deputy CTO, Azul. “Azul leads more OpenJDK projects and supports more Java versions than any other vendor, including Oracle. Having Azul’s builds of OpenJDK on the Adoptium Marketplace along with other distributions gives users the widest choice of runtime accessible from one convenient location.”

Huawei

“Huawei BiSheng (JDK 8) is an open source high-performance OpenJDK distribution that can be used in production environments, with optimized performance and enhanced stability on the ARM architecture,” said Shanyao Chen, Huawei BiSheng (JDK 8) Senior Project Manager. “We appreciate the Adoptium working group building a great marketplace for users to obtain high-quality, open source distributions, and Huawei BiSheng (JDK 8) is proud to be part of it!”

IBM

“IBM is proud to make available at the Adoptium Marketplace the IBM Semeru Runtime Certified Edition that includes both the Eclipse OpenJ9 JVM and OpenJDK class libraries,” said Theony Mousa, director of Engineering, IBM Application Platform and Runtimes. “Our commitment to producing only high quality, secure, and performant Java SE runtimes is firmly echoed in the guiding principles of the Adoptium Marketplace. With Adoptium's AQAvit quality verification testing, the Java community can trust that the IBM Semeru Runtime Certified Edition will reliably run all Java workloads.” 

Microsoft

“Microsoft has a long history of supporting developer ecosystems, and we are glad to be a part of the Working Group that is launching the Eclipse Foundation's Adoptium marketplace. The Adoptium marketplace will provide a vendor-neutral home where business-ready, Java-compliant OpenJDK binaries are produced and tested by the AQAvit quality suite. It offers Java developers, users, and deployers a trusted source for OpenJDK binaries from multiple vendors with no restrictions,” said Amanda Silver, CVP of Product, Developer Division, Microsoft. “Alongside the other founding members, Microsoft looks forward to contributing to the Adoptium marketplace with the Microsoft Build of OpenJDK, a free OpenJDK distribution that is optimized for and supported on Azure.“

Red Hat

“At Red Hat we have a long tradition of working with communities to ensure open source software is ready for Enterprise IT use,” said Mark Little, VP, Engineering, Red Hat. “The Adoptium marketplace is a welcome innovation that supports our mission, and we are committed to participating in the working group to define the open AQAvit standard. We are delighted to bring Red Hat’s build of OpenJDK to the marketplace, demonstrating that it meets the expectations of modern application developers.”

Alibaba Cloud

“Alibaba Cloud has a strong record of open source and community involvement. As a founding member of the Adoptium Working Group, we are thrilled to support the launch of Adoptium Marketplace,” said Sanhong Li, director of Compiler & Runtime, Alibaba Cloud Intelligence. “The Adoptium Marketplace is a trusted place where developers can obtain high quality, production-ready binaries of OpenJDK builds. We look forward to participating in the Adoptium marketplace with the Alibaba Dragonwell, a downstream version of OpenJDK optimized for online e-commerce, financial, logistics applications, and numerous other distributed services on Alibaba Cloud.”

About the Eclipse Foundation

The Eclipse Foundation provides our global community of individuals and organizations with a mature, scalable, and business-friendly environment for open source software collaboration and innovation. The Foundation is home to the Eclipse IDE, Jakarta EE, and over 400 open source projects, including runtimes, tools, and frameworks for cloud and edge applications, IoT, AI, automotive, systems engineering, distributed ledger technologies, open processor designs, and many others. The Eclipse Foundation is an international non-profit association supported by over 330 members, including industry leaders who value open source as a key enabler for their business strategies. To learn more, follow us on Twitter @EclipseFdn, LinkedIn or visit eclipse.org.

Third-party trademarks mentioned are the property of their respective owners.

###

Media contacts

Schwartz Public Relations for the Eclipse Foundation, AISBL

Stephanie Brüls / Susanne Pawlik

Sendlinger Straße 42A

80331 Munich

EclipseFoundation@schwartzpr.de

+49 (89) 211 871 – 64 / -35

Nichols Communications for the Eclipse Foundation, AISBL

Jay Nichols

jay@nicholscomm.com

+1 408-772-1551

On behalf of the Eclipse JKube team and everyone who has contributed, I'm happy to announce that Eclipse JKube 1.8.0 has been released and is now available from Maven Central.

Thanks to all of you who have contributed with issue reports, pull requests, feedback, and spreading the word with blogs, videos, comments, and so on. We really appreciate your help, keep it up!

What's new?

Without further ado, let's have a look at the most significant updates:

• Graduation of Kubernetes and OpenShift Gradle plugins
• Startup Probe definition is now possible via XML or DSL configuration
• Fragments can be picked from multiple environment directories
• Support for JakartaEE and JavaEE based projects with our WebApp generator
• Better support for Apple M1
• Group and Version labels are now configurable
• ğŸ�› Many other bug-fixes and minor improvements

Graduation of Kubernetes and OpenShift Gradle plugins

Our Gradle plugins (Kubernetes Gradle Plugin & OpenShift Gradle Plugin) have finally graduated and are out of preview. You should expect full support as compared to the Maven plugins.

Fragments can be picked from multiple environment directories

So far, it was possible to use a different set of fragments depending on the environment you were targeting. This is useful in case you want to provide different configurations or resources for environments such as development, staging, production, and so on. However, it was not possible to have common resources or fragments that could be shared among the different environments.

Starting on this release, it's now possible to load fragments from multiple directories. Considering a directory structure like the following:

src
└── main
    └── jkube
        ├── common
        ├── dev
        └── prod

You can setup two different profiles that share the resources in the common directory:

<profiles>
  <profile>
    <id>dev</id>
    <properties>
      <jkube.environment>common,dev</jkube.environment>
    </properties>
  </profile>
  <profile>
    <id>prod</id>
    <properties>
      <jkube.environment>common,prod</jkube.environment>
    </properties>
  </profile>
</profiles>

Support for JakartaEE and JavaEE based projects with our WebApp generator

The base image for our WebApp generator is now using Tomcat 10 by default. Tomcat 10 supports JakartaEE based projects. However, we deploy the project in the webapps-javaee directory, which automatically migrates any JavaEE project to JakartaEE if necessary. You can read more about this in the Tomcat 10 migration guide.

Check the following video for a live demo and additional details:

Using this release

If your project is based on Maven, you just need to add the Kubernetes Maven plugin or the OpenShift Maven plugin to your plugin dependencies:

<plugin>
  <groupId>org.eclipse.jkube</groupId>
  <artifactId>kubernetes-maven-plugin</artifactId>
  <version>1.8.0</version>
</plugin>

If your project is based on Gradle, you just need to add the Kubernetes Gradle plugin or the OpenShift Gradle plugin to your plugin dependencies:

plugins {
  id 'org.eclipse.jkube.kubernetes' version '1.8.0'
}

How can you help?

If you're interested in helping out and are a first-time contributor, check out the "first-timers-only" tag in the issue repository. We've tagged extremely easy issues so that you can get started contributing to Open Source and the Eclipse organization.

If you are a more experienced developer or have already contributed to JKube, check the "help wanted" tag.

We're also excited to read articles and posts mentioning our project and sharing the user experience. Feedback is the only way to improve.

Project Page | GitHub | Issues | Gitter | Mailing list | Stack Overflow

The support for textual programming languages is excellent in modern, web-based IDEs, such as Eclipse Theia and VS Code. The hugely...

The post Diagram Editors in Cloud IDEs appeared first on EclipseSource.

Learn how Smalltalk and Java both influenced Eclipse Collections and can help you grow as a developer.

Taking a walk with Smalltalk

On May 12, 2022, I gave a talk titled “Smalltalk or Java? Why not both!” at the Pittsburgh Java User Group. The meetup was hosted by Sirisha Pratha and Rinat Gatyatullin. The talk was recorded and is available now on the Pittsburgh JUG YouTube channel(video linked below). I demoed some language and library features switching back and forth between Smalltalk and Java/Eclipse Collections code examples using Pharo 10.0 and IntelliJ IDEA 2022.1.

The talk was a lot of fun for me. It’s been 22 years since I programmed in Smalltalk professionally. I was asked by an attendee which Smalltalk I used previously. I answered that I programmed in IBM VisualAge for Smalltalk. I also mentioned that I started learning Java using IBM VisualAge for Java, which was implemented in VisualAge for Smalltalk. VisualAge for Java would eventually evolve into the Eclipse IDE, which was open sourced by IBM over 20 years ago.

Four decades and counting as a developer

I’ve been programming for 40 years. Here’s a list of most of the languages I have programmed in.

3 Decades, 3 Languages and 3 Lambda Syntaxes

Over the past 30 years, I have programmed professionally in three different languages that have had a lasting impact in my thought process as a developer. All three of these languages have support for lambdas.

During the talk, I explained I could never see myself programming in a language without support for lambdas again.

Blog References

I referenced the following blog links during the talk. The code I shared in the live coding sessions are available in the blogs. I also explain in some detail how to get up and running using the Pharo Smalltalk IDE.

Smalltalk

• A little Smalltalk for the soul
• Exploring the Smalltalk Collections API in unit tests using Pharo 8.0

Java with Eclipse Collections

• Blog Series: The missing Java data structures no one ever told you about
• My 25 favorite methods from the Eclipse Collections API

Both Smalltalk and Java

• Java Streams are great but it’s time for better Java Collections
• Comparing my Smalltalk and Eclipse Collections Wordle Kata Solutions

Eclipse Collections = Smalltalk + Java

Eclipse Collections is a bridge built between two amazing programming worlds. Here’s a table that explains the influence both programming languages have had on the evolution of the now 18 year old open source Java library.

My Pharo Playground from the talk

The following is a snapshot of the Pharo playground I used as a REPL to highlight and execute code. The output in the Transcipt in the image is the result of executing the highlighted code. I noted how Interval is inclusive in Smalltalk, and when you use a fractional step by, Smalltalk automatically reduces fractions when it can. I mentioned that we have Interval, IntInterval and LongInterval in Eclipse Collections.

Learn Multiple Languages

Learning multiple languages can help us grow as developers. There are a lot of programming languages to learn out there. I believe there is as much value, if not more, in learning classic languages like Lisp and Smalltalk as there is in learning the shiny new programming languages. The classic languages like Lisp and Smalltalk continue to evolve and solve new problems. In preparing for this talk, I learned some new things I didn’t know about Smalltalk as well as some cool features in the Pharo Smalltalk IDE. The Collections API in Pharo Smalltalk seems to be continuing to evolve. I hope by giving more talks like this, the Java and Smalltalk communities can learn about new things that we have implemented in Eclipse Collections.

I hope to implement a Lazy Iterable library for Pharo Smalltalk in the future. I will open source the library of course, once I get around to it.

Sharing is caring.

I will post a link to the video from the talk here once it is available on YouTube. Stay tuned!

Update: The Video

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Smalltalk or Java? Why not both! was originally published in Javarevisited on Medium, where people are continuing the conversation by highlighting and responding to this story.

In a previous post I showed how to use the import namespace feature of Xtext. This feature helps to easily implement java like imports. However, Xtext also supports imports more in a C-like style, i.e. imports where a whole file imported, and its definitions made available to the current file.

For example, in our block language, it should be possible to import the name space from another Block and then use in our alias. We can for example define Block3 in file Block3.block

block Block3 {
		field myField1
		field myField2
}

And then use it in the file Block1.block as follows:

 import "Block3.block"

block Block1 {

	alias myAlias1 aliases Block3.myField1
}

The block language with import name space support

First, we modify the grammar to include an import statement:

grammar com.idiomaticsoft.dsl.block.Block with org.eclipse.xtext.common.Terminals

generate block "http://www.idiomaticsoft.com/dsl/block/Block"

Model:
	(imports+=Import)*
	blocks+=Block*;

Import:
	'import' importURI=STRING
;

Block:
	'block' name=ID'{' (members+=Member)* '}';

Member:
	Block | Field | Alias;

Field:
	'field' name=ID;

Alias:
	'alias' name=ID 'aliases' alias=[Member|MemberFQN];

MemberFQN:
	ID ("." ID)*;

This grammar includes a new Import rule that has importURI attribute. This attribute is one of those Xtext defaults, and to make the infrastructure work, you need to name your attribute like that.

The next step is to modify the runtime module to bind the right global scope provider. To do this, we modify the file as follows:

package com.idiomaticsoft.dsl.block;

import org.eclipse.xtext.scoping.IGlobalScopeProvider;
import org.eclipse.xtext.scoping.impl.ImportUriGlobalScopeProvider;

/**
 * Use this class to register components to be used at runtime / without the
 * Equinox extension registry.
 */
public class BlockRuntimeModule extends AbstractBlockRuntimeModule {

	@Override
	public Class<? extends IGlobalScopeProvider> bindIGlobalScopeProvider() {
		return ImportUriGlobalScopeProvider.class;
	}

}

With theses two modifications, the scoping will work. Nevertheless, the import will not show an error when the file is not found. This can be corrected by modifying the MWE file to enable the use the ImportUriValidator, this is done as follows:

module com.idiomaticsoft.dsl.block.GenerateBlock

import org.eclipse.xtext.xtext.generator.*
import org.eclipse.xtext.xtext.generator.model.project.*

var rootPath = ".."

Workflow {
	
	component = XtextGenerator {
		configuration = {
			project = StandardProjectConfig {
				baseName = "com.idiomaticsoft.dsl.block"
				rootPath = rootPath
				runtimeTest = {
					enabled = true
				}
				eclipsePlugin = {
					enabled = true
				}
				eclipsePluginTest = {
					enabled = true
				}
				createEclipseMetaData = true
			}
			code = {
				encoding = "UTF-8"
				lineDelimiter = "\n"
				fileHeader = "/*\n * generated by Xtext \${version}\n */"
				preferXtendStubs = false
			}
		}
		
		
		
		language = StandardLanguage {
			name = "com.idiomaticsoft.dsl.block.Block"
			fileExtensions = "block"
			serializer = {
				generateStub = false
			}
			validator = {
				composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
				composedCheck = "org.eclipse.xtext.validation.ImportUriValidator"
				generateDeprecationValidation = true
			}
			generator = {
				generateXtendStub = true
			}
			junitSupport = {
				junitVersion = "5"
			}
		}
	}
}

Things to consider

The default for this feature are the following:

1. If you want to have several importURI, then you need to use several Import rules. You cannot have a rule like importedNamespace+=ImportFQN. The default implementation expects this attribute to be a string, not a list.
2. The imports of the rules are valid for all elements in the container and its children.

The full code for this example can be found here.

OTTAWA, CANADA – May 12, 2022 - The OpenHW Group, a non-profit, member-driven organization transforming microprocessor technology development through an open-source ecosystem, today announced the addition of two regional working groups.

The new groups, the OpenHW Asia Working Group (AWG) and OpenHW Europe Working Group (EWG) address regional open hardware requirements, technical project incubation, and data governance. With participation from key regional organizations and thought leaders they aim to foster a global, unified open hardware ecosystem while providing focus on specific Asian and European ecosystem considerations. 

“The open-source methods that have reshaped the software industry are now substantially transforming microprocessor technology development, licensing, and implementation”, said Rick O’Connor, CEO, OpenHW Group. “The OpenHW Group is a major driver of this shift and continues to create a vibrant and worldwide OpenHW ecosystem.  OpenHW EWG and OpenHW AWG bring insight into regionally focused project opportunities to our Technical Working Group. Constituted as Eclipse Foundation Working Groups, they bring a truly global footprint to OpenHW Group, taking into account the digital sovereignty requirements of our global membership.”   

The New Frontier of Open Source

The availability of the free and open RISC-V ISA has unleashed a new frontier of innovation in the processor design community.  However, the RISC-V ISA specification covers the ISA itself and does not provide tangible processor designs. That’s where the OpenHW Group comes in. OpenHW Group members design and verify RISC-V processor cores (the CORE-V family) to industrial quality levels and release the tested processor code as open-source artifacts.

The OpenHW Group worldwide membership is developing the CORE-V family of open-source, RISC-V microprocessor cores, along with associated accelerators, interfaces, enabling hardware and SoC platforms, and software toolchains in a comprehensive ecosystem approach.  Starting from a single project in 2019, OpenHW Group now fosters over twenty open-source projects.  In OpenHW Group’s model, industry, academic, and individual contributors drive innovation within a collaborative framework. Each participant helps to advance projects according to their priorities and skillsets, reducing development costs and fostering an ecosystem of stakeholders with a shared interest in bringing open-source projects to fruition. 

 The European and Asian Working Groups will complement OpenHW Group’s existing Technical Working Group and the Cores, Verification, Hardware and Software Task Groups. All OpenHW working groups are directed by members and are open to participation from all OpenHW Group members.

For more information about the OpenHW Group, explore our projects or contact us at info@openhwgroup.org.

About OpenHW Group and CORE-V family 

The charter of the OpenHW Group is to serve developers of processor cores and hardware and software engineers who design SoCs with greater awareness, understanding and availability of open-source processor implementations for use in high volume production. OpenHW provides an infrastructure for hosting high quality open-source HW developments in line with industry best practices. The cores task group within the organization has the mandate to develop feature and functionality roadmap and the open-source IP for the cores within the OpenHW Group such as the CORE-V Family of open-source RISC-V processors. The OpenHW Group, a global, non-profit, member-driven organization in Canada, and partnered with the Eclipse Foundation.

Media Contact for OpenHW Group

Michelle Clancy, Marketing Director, OpenHW Group

+1.503.702.4732 michelle@openhwgroup.org

As everyone who is involved in the software industry is well aware, security is a significant topic these days. In particular, open source supply chain security is top of mind across the entire ICT industry. The Eclipse Foundation, its community, its projects, and its working groups all have a strong motivation to be leaders in advocating and implementing security best practices. Our members, adopters, users, and stakeholders all desire that their security risks be mitigated to the degree possible. 

One thing that is clear, however, is that simply putting the burden of added security work on the shoulders of our committers and project leaders is not an option. This topic needs to be addressed by services provided by the Eclipse Foundation to our project community or it will fail. Without strong support in terms of release and build engineering, tooling, and education, developers simply do not have the time, interest, or skills necessary to be responsible for implementing security best practices. It is equally true that security, and particularly supply chain security, requires a programmatic approach. Security is not an attribute that you simply add to existing software.

So we need to provide services to our projects to implement our Open Source Software Supply Chain Best Practices. We envisage this as a collection of services provided to our projects by staff to protect our code repositories, secure third party artifacts, provide security audits, secure build pipelines, and protect build outputs. 

The Eclipse Foundation has long had a security policy, and is a CVE numbering authority. We have a long track record of taking security seriously. However, we are not going to be able to accomplish more without leadership. So, to that end, I am very pleased to announce that we have recently promoted Mikaël Barbero as our new Head of Security. Mikaël is well known to our community as having led our Common Build Infrastructure for many years, as well as having authored the best practices document referenced above. Mikaël will be providing leadership to our security initiatives, and will be working closely with our projects and our IT staff to steadily improve security across the Eclipse community. Some of this work will complement or leverage related efforts to improve our IP processes and provide software bill of materials (SBOMs) for all of our projects. We expect to make a number of program announcements over the coming months, so stay tuned. Please join me in welcoming Mikaël in his new role.

Solving a problem with multiple languages and libraries can sometimes bring new insights.

Challenge Yourself and Others to Learn

I accepted a JLDD (Jet Lag Driven Development) challenge recently to implement a Wordle Checker Kata in Java. My friend José Paumard sent me this challenge and I set out to solve it using my favorite Java collections framework — Eclipse Collections. I wrote a blog about the challenge and my Java / Eclipse Collections solution.

A Wordle JLDD Kata Challenge

José then published his response in a great JEP Café tutorial video which he tweeted about.

 — @JosePaumard

I really enjoyed the video and learned an approach to solving this problem using pattern matching that I wouldn’t have thought of on my own.

I had come up with three slight variations on the same solution using Java with Eclipse Collections, and thought it might be interesting to try and build a solution using Pharo Smalltalk. So of course I built one, and tweeted about it.

 — @TheDonRaab

Smalltalkers will hopefully recognize the emoji picture I used. It is the basic syntax of a lambda in Smalltalk — [|]. Parameters go on the left side of the pipe, and statements/expressions go on the right.

A few days after posting this, I posted a slight variation on the Smalltalk solution. I have included both Smalltalk solutions in the blog along with three Java / Eclipse Collections solutions.

My Smalltalk Solutions

I used Pharo 8.0 to implement my Smalltalk solution. First, I wrote the set of test assertions in a WordleTest class. Each assertion creates an instance of a Wordle class with a hidden word, and then calls a method named guess: which will output the expected result of the two words matched.

I created a Wordle class with an instance variable named hidden, and then added a method named guess: which takes a String parameter.

As you read my Smalltalk solutions, I would note the following. Smalltalk has a very simple and powerful programming paradigm which leads to a very minimal syntax with only five reserved words (true, false, nil, self, super) in the language. There are no control statements (if, for, while) in Smalltalk , and everything is accomplished by sending messages to objects. The equivalent of if statements, for loops, while loops, etc. in other languages are accomplished by sending messages to different objects. Much of this is accomplished by passing lambdas around to methods as parameters, or calling appropriate methods on lambdas.

Smalltalk Solution #1

I wrote my first Smalltalk solution for guess: using with:do: and with:collect: methods. This solution passes all of the tests above.

The with:do: and with:collect: methods are available on the SequenceableCollection class in Smalltalk. In Smalltalk, a String is a SequenceableCollection. The following is a view of the Smalltalk Collection hierarchy.

The method named with:do: lets you iterate over two String instances at the same time. The first argument passed is the the String that you want to iterate over with this String, and the second argument is a two argument lambda which doesn’t return anything (equivalent of a Procedure2 or BiConsumer in Java). This let me compare letters at the same index, exclude all of those that were equal and add the remaining letters to the Bag named remaining, which I initialized on the first line of the method. For anyone unfamiliar with the Bag data structure, a Bag is an unordered collection that allows duplicates. Nikhil Nanivadekar wrote a blog about the Bag data structure in Eclipse Collections which explains how it is implemented.

The second method, with:collect:, is similar to with:do: in that it iterates over both String instances together. The difference is that the two argument lambda that is passed as the second argument is used as a Function2 (a two argument Function or BiFunction in Java) to transform the characters to a new String. The transformation will result in either an uppercase letter, a lowercase letter or a “.” being returned for each combination of letters.

The ifTrue:ifFalse: method is a method on the Boolean class in Smalltalk and is overridden by the True and False subclasses. The method takes two lambdas as arguments. Depending on whether the Boolean is true or false, one of the two lambdas will be evaluated. The = method tests for equality and will return true or false. The literal true is the single instance of the class True , and false is an instance of False. I always thought it was kind of clever how Smalltalk implemented if control flow logic without using statements.

Finally, the remove:ifAbsent: method is available on most Collection classes in Smalltalk, and in this instance I am calling this method on a Bag. The result of the method is to return the object that is removed, or if the object is not in the collection, evaluate and return the result of the lambda.

Smalltalk Solution #2

For the second Smalltalk solution, I had hoped to find a method named with:reject:, but unfortunately, no such method exists today. I could have added the myself, but instead I decided to use a method that was already implemented. I used withIndexSelect: as a somewhat more awkward alternative to with:reject: but at least more terse alternative to using with:do:. I also formatted the code for with:collect: using the recommended auto-formatting in Pharo. It should make it easier to read and is a good way to break up the ifTrue:ifFalse: message send with the nested lambdas.

The withIndexSelect: method is passed each character in the hidden String along with its corresponding index in the String. Indexes in Smalltalk are one based, so start at 1, not 0. This can’t be seen in this particular code example, but I think it’s an interesting difference between Smalltalk and Java. I look up the character at the same index position in the guess String using the at: method. Because I am using select: (inclusive filter) and not reject:(exclusive filter), I have to negate the comparison. I use ~= to compare the two letters, which translates to “not equal” in Smalltalk. Finally, I convert the resulting String instance from the call to withIndexSelect: to a mutable Bag. The rest of the solution is identical to my first Smalltalk solution.

Update: May 10, 2022

I upgraded my Pharo IDE to version 10.0. I also refactored my test cases to use a shared data set. This reduced a lot of duplication, and made the test cases much more pleasant to read.

I put the shared test data in a method named hiddenGuessExpectedData.

I renamed the test methods to testGuess and testGuessWithIndexSelect.

A neat side effect of renaming the test methods to better match the name of the methods on the class, is the class now displays green spheres next to each method and allow you to run the tests for the methods in the class itself.

My Eclipse Collections Solutions

All of my Java solutions used features of the Eclipse Collections library. All three solutions used Strings.asChars(string) to provide access to the primitive collection API. In Eclipse Collections, this static method returns a CharAdapter instance wrapping the String.

Two of the three Eclipse Collections solutions use methods that end in WithIndex. The third solution uses a method named zipChar which allows two primitive collections to be zipped together into char pairs. The basic strategy is pretty much the same between my three solutions, with minor differences in implementation.

• Create a Bag of characters that do not have direct positional matches between both String instances (hidden and guess)
• Collect up one of three possible states for each character as a result String. Convert exact positional letter matches to uppercase, letter match in different position to lowercase, and no letter match to a “.”

Eclipse Collections Solution # 1

The first solution I wrote using injectIntoWithIndex and collectWithIndex. I wrote the original Wordle JLDD Kata Challenge blog with this solution included. You can read the blog for more details on the solution.

The thing that bothered me most about this solution is that Eclipse Collections did not yet have rejectWithIndex available on primitive collections. The solution using injectIntoWithIndex is really implementing the rejectWithIndex pattern.

Eclipse Collections Solution #2

I have recently contributed selectWithIndex and rejectWithIndex methods to primitive collections in Eclipse Collections. The same methods existed on the object collection API in Eclipse Collections since the 11.0 release. I used rejectWithIndex in my second solution, along with the collectWithIndex approach used in Eclipse Collections Solution #1.

The rejectWithIndex method here takes a target collection to store the results in.

Eclipse Collections Solution #3

My third Eclipse Collections solution using zipChar and collectChar was inspired by an alternative zipChar solution written by Vladimir Zakharov.

The method zipChar creates a List of character pairs between both String instances. I reject the exact positional matches between char pairs and collect those char values into a MutableCharBag. Then I collect char values into a String based on the algorithm described above.

From Wordle Kata Solutions to EC Integration Test

The Eclipse Collections Wordle Kata solutions I came up with are good examples of using some of the String iteration patterns available in the library. Because I am leveraging multiple String iteration patterns, I thought it made sense to turn the kata solutions into an integration level test for Eclipse Collections. I committed a new WordleTest class into Eclipse Collections along with my contribution of the primitive versions of selectWithIndex and rejectWithIndex. Below is the source code for the test.

eclipse-collections/WordleTest.java at master · eclipse/eclipse-collections

Comparing the solutions

Here is a visualization of the intersection and set of differences between the methods available in Smalltalk and Eclipse Collections to solve this particular kata challenge.

I quite liked the with:do: and with:collect: methods in Smalltalk. I would have preferred to have a with:reject: method for this particular use case. While I could have added this method on my own to SequenceableCollection, it would be nice if it was part of the standard library. However, I doubt that this particular use case will be incentive enough to have it added. The following are the methods using with: as a prefix that are available on SequenceableCollection.

It was interesting to see that there is a withIndexSelect: method in Smalltalk which is a slight difference in name to the equivalent of the selectWithIndex method in Eclipse Collections. There is no equivalent of rejectWithIndex, which is my preference in this use case. The methods collectWithIndex: and withIndexCollect: both exist in Pharo Smalltalk and are synonyms. It appears the naming preference is now in favor of withIndex as a prefix for these methods.

The method zipChar in Eclipse Collections has no equivalent in Pharo Smalltalk today. There is also no equivalent of asLazy either. Most iteration methods on the Smalltalk Collection classes are eager.

In terms of elegance, I think the with:do and with:collect: methods worked the best for this use case. Again, it would have been slightly simpler and clearer if there was a with:reject: method.

In terms of flexibility, I think the withIndex solutions are better than both the other solutions. Since you have the index, you can do whatever you need with it. For example, this would allow you to iterate over as many collections simultaneously as you need.

In terms of fluency and ease of writing, I really liked the asLazy plus zipChar solution which could then just use reject, collectChar and toBag.

Here’s a link to the tweet where Vladimir Zakharov shared his zipChar solution using Eclipse Collections.

 — @zakhav

Summary

Learning multiple programming languages and libraries and using them to solve various coding katas can help you improve as a programmer. I think it’s interesting to see how languages and libraries compare and how they evolve over time. I don’t recall the methods I used in Pharo Smalltalk being available in VisualAge for Smalltalk when I programmed in it in the 1990s. This is a good sign, as it means the language and core libraries are still evolving. Similarly, the Eclipse Collections library continues to evolve as Java does, and new methods like selectWithIndex and rejectWithIndex now available on both object and primitive collections are a good example of this.

Thank you for reading! I hope you enjoyed this Wordle JLDD Kata Challenge comparison with Smalltalk and Eclipse Collections.

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

The Eclipse Foundation Community Awards are back! The Eclipse Foundation has a vibrant community of many dedicated individuals, who deserve to be recognized for their contributions. Submit a nomination before May 31. 

We are happy to announce the Eclipse Theia 1.25 release! This release contains 48 merged pull requests and we welcome eight...

The post Eclipse Theia 1.25 Release: News and Noteworthy appeared first on EclipseSource.

Discover a better way to count things in Java.

Using Map to count the beads on an abacus

Here’s a simple problem to solve using a Map. How can we count the number of beads tallied on an abacus by color? First, I will define an enum for Color.

Next, I will create a HashMap<Color, Integer> and use the merge method to add and subtract values for a specific colors. I will leave one color bead (Color.RED) out from the counting. Then I will assert the result of querying the Map for the counts of the colors using Map.get.

I use the same merge operation for adding and subtracting. I use negative Integer values to subtract. I use the method reference Integer::sum for the BiFunction parameter the merge method takes to determine how to merge values. Note, that even though I am using primitive literal int values here, they are auto-boxed as Integer objects in the Map., and the Integer::sum call will result in more unboxing and boxing of Integer objects.

In the final assertion in the test, the call to get with the Color.RED instance returns null. I would have to explicitly set the value to 0 for it to come back as 0 when calling get. One way to address the null return problem with get, is to use getOrDefault instead. This will allow you to return a default value in the case that the key does not exist in the Map.

The merge and getOrDefault methods were both added in Java 8, and are useful methods for developers to understand on Map.

Using a primitive Map instead of a Map

In Eclipse Collections, there is support for primitive Maps. There is an ObjectIntMap type and corresponding implementation that can be used to count the beads. I will use a MutableObjectIntMap so I can mutate the map and create the implementation using the ObjectIntMaps factory class.

A MutableObjectIntMap has an addToValue method that can be used to put a new value in the map, or add to an existing value in a map. The method getIfAbsent can be used to look up the values. The 0 value is an int, not an Integer object, and none of the calls to addToValue result in any boxing.

Note: Eclipse Collections HashBag class leverages a MutableObjectIntMap.

Using Bag to count the beads on an abacus

So how would I solve this problem using a MutableBag from Eclipse Collections? A MutableBag is a Collection, not a Map. It has add and remove methods which I can use to add single items. MutableBag also has addOccurrences and removeOccurrences methods which allow adding a specific number of items at once.

The individually named methods on Bag are more intention revealing than the single method named merge for this use case. Notice that in the case of Color.RED, the Bag will return 0 from the call to occurrencesOf. Bag does not suffer from the null problem that Map.get does. If a value does not exist in the Bag, the occurrencesOf method always returns 0.

More Information

There are a few more blogs which provide more details on the Bag type and supporting methods like countBy in Eclipse Collections. The following blog was written by Nikhil Nanivadekar. It describes many of the implementation details of the Bag type in Eclipse Collections, that can result in both memory and performance benefits.

Bag — The Counter

I also wrote a separate blog describing the countBy method in Eclipse Collections which provides Bag as its return type.

EC by Example: CountBy

I hope you enjoyed reading this blog and learned something useful about Map and Bag types in the process!

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Other Java Articles you may like

• The 2022 Java Programmer RoadMap
• 10 Best Places to Learn Java Online for Free in 2022
• 13 Topics You Should Prepare for Your Next Spring Boot Interview in 2022

Map vs. Bag was originally published in Javarevisited on Medium, where people are continuing the conversation by highlighting and responding to this story.

Today we celebrate the one year anniversary of the Eclipse IDE Working Group. A year ago, the Eclipse Foundation launched this Working Group focused on the Eclipse IDE and the Eclipse Simultaneous Release (SimRel). We would like to share some of our successes since the launch of the Working Group in April 2021. 

Highlights

• 20 Years of the Eclipse IDE: 2021 was a momentous year for the Eclipse IDE, as we celebrated its 20 year anniversary!
• Welcome Ed! Ed Merks joined us as a SimRel Architect and Release Engineer. Ed’s first set of tasks included preparing PGP signing support for the 2022-06 release and mapping out the project dependencies.
• Productive collaboration Our collaboration with the Planning Council has been very effective. We have identified the top issues as outlined by the Planning Council and have a plan to address them.

PGP: A Community Success Story

A great community success story for the Eclipse IDE Working Group is the delivery of a fully-functional, secure PGP implementation for Eclipse 4.23 (SimRel 2022-03). This enhancement augments Eclipse’s existing security support which is based on jar signing. Jar signing has the significant drawback that artifacts originating from external dependencies must be modified in order to sign them, i.e., jar signatures are intrinsic to the artifact. In contrast, PGP signatures are extrinsic to the artifact and have long been used in Maven repositories to provide certification of origin. Eclipse’s PGP support facilitates significantly streamlined consumption of Maven-based artifacts by Eclipse projects, making it easier for our community to exploit and deliver the latest and greatest libraries with each quarterly simultaneous release.

The initial proof-of-concept PGP implementation was contributed by Mickael Istria. In combination with Mickael’s on-going participation, along with Christoph Läubrich’s technical insights, the working group has helped to harden the PGP implementation to industrial-strength quality for the SimRel 2022-03 delivery. Even the existing support for jar signing has been improved, as users can now easily save trusted X.509 certificates to avoid repeated trust prompts as is typical with self-signed certificates. Issue 11 provides a detailed track record of all the activities around PGP signing during the 2022-03 release cycle as well as additional background information.

With this groundwork in place, our community as a whole can exploit PGP signing for broader adoption in the Eclipse 4.24 (SimRel 2022-06) delivery this coming June.

Planning Council’s Top 3 Items

The Planning Council plays an important role in the Eclipse IDE WG. The Planning Council can be seen as the “technical” arm of the WG. At the beginning of the first year, under the leadership of Mélanie Bats, the Planning Council was tasked by the Steering Committee to identify the top issues affecting the successful release and adoption of the Eclipse IDE as a platform and a product.

After much brainstorming and debate the Planning Council recommended the “top three” items to the Steering Committee to focus on:

• The “Bus Factor“, particularly of the release engineering processes of the Simultaneous Release (SimRel).
• Identifying individual project risks, for example identifying which projects contributing to the SimRel are under-resourced and understanding which downstream projects are affected.
• Updating the graphical layer Eclipse where it is lagging behind operating system changes, for example improving dark mode, better operating system and web browser integration.

The Steering Committee took these items and translated them to action points that are now being carried out and has allocated a substantial portion of the IDE WG’s budget to improving these common parts of the Eclipse IDE. The highlights of this work include:

• Hiring Ed Merks as the SimRel release engineer.
• Ed has also found time to start mapping out the incredibly complicated dependency graphs between the dozens of different projects contributing to the SimRel to better understand the impact of any particular project discontinuing participation and to fully understand the dependency chain of each bundle in the SimRel repository.
• The Eclipse Foundation has created new guidelines for funding work such as the graphical layer improvements. This is the most recent action point and already some bugs are being fixed under this program.

One year into the Eclipse IDE WG and Jonah Graham is now the chair of  the Planning Council. The Planning Council is pleased to see some concrete actions taking place under the new Working Group. The structures and processes in the Working Group have progressed well and additional funding of the IDE WG will see direct improvements in the quality, stability and adoption of the Eclipse IDE.

Engage in the Working Group

We still have much to do! If you are interested in joining us and supporting the development of the Eclipse IDE technology stack, improving the user experience of the platform, and making it more attractive for organisations, let us know. We welcome the opportunity to speak with everyone who wants to help shape the future of the Eclipse IDE.

The resources funded by the IDE Working Group members are really paying dividends for our community, both for the producers and for the consumers. If you’re a consumer, please consider investing in our community’s ongoing success by supporting the Eclipse IDE with funding, contributions, new ideas, new points of view, and by getting directly involved in development efforts. Better funding enables us to achieve more, and more hands make the work lighter.

If you are interested in becoming a Working Group member, you can get in touch with us by completing the membership form or by sending an email to the Membership Coordination Team. If you are interested in becoming a sponsor, let us know.

Stay In Touch

We love exchanging ideas, so if you have any questions or would like to know more about what we do here, connect with us!  You can also join our meetings to find out more about what we’re up to. They are open to the community and take place every 2 weeks from 2:30pm to 3:30pm (CET) on Tuesdays. Or you can contact a member of the steering committee, we’re always happy to talk to you.

I had previously written about mirroring Eclipse p2 repositories (see this blog tag), but I’ll show how to do that with Tycho and one of its plugins in this post.

The goal is always the same: speed up my Maven/Tycho builds that depend on target platforms and insulate me from external servers.

The source code of this example can be found here: https://github.com/LorenzoBettini/tycho-mirror-example.

I will show how to create a mirror of a few features and bundles from a few p2 repositories so that I can then resolve a target definition file against the mirror. In the POM, I will also create a version of the target definition file modified to use the local mirror (using Ant). Moreover, I will also use a Tycho goal to validate such a modified target definition file against the local mirror. The overall procedure is also automatized in the CI (GitHub Actions). This way, we are confident that we will create a mirror that can be used locally for our builds.

First of all, let’s see the target platform I want to use during my Maven/Tycho builds. The target platform definition file is taken from my project Edelta, based on Xtext.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?pde version="3.8"?>
<target name="edelta.target" sequenceNumber="13">
<locations>
<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="org.eclipse.equinox.executable.feature.group" version="0.0.0"/>
<unit id="org.eclipse.jdt.feature.group" version="0.0.0"/>
<unit id="org.eclipse.platform.feature.group" version="0.0.0"/>
<unit id="org.eclipse.pde.feature.group" version="0.0.0"/>
<unit id="org.eclipse.sdk" version="0.0.0"/>
<unit id="org.eclipse.emf.sdk.feature.group" version="0.0.0"/>
<unit id="org.eclipse.swtbot.forms.feature.group" version="0.0.0"/>
<unit id="org.eclipse.swtbot.ide.feature.group" version="0.0.0"/>
<unit id="org.eclipse.swtbot.eclipse.feature.group" version="0.0.0"/>
<repository location="https://download.eclipse.org/releases/2022-03"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="org.eclipse.emf.mwe2.launcher.feature.group" version="0.0.0"/>
<repository location="https://download.eclipse.org/modeling/emft/mwe/updates/releases/2.12.2/"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="org.eclipse.xtext.sdk.feature.group" version="0.0.0"/>
<repository location="https://download.eclipse.org/modeling/tmf/xtext/updates/releases/2.26.0/"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="de.itemis.xtext.antlr.feature.feature.group" version="2.1.1.v201405091103"/>
<repository location="https://download.itemis.com/updates/releases/2.1.1/"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="org.eclipse.epsilon.picto.feature.feature.group" version="0.0.0"/>
<unit id="org.eclipse.epsilon.emf.feature.feature.group" version="0.0.0"/>
<repository location="https://download.eclipse.org/epsilon/updates/2.4/"/>
</location>
<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
<repository location="https://download.eclipse.org/tools/orbit/downloads/2022-03/"/>
</location>
</locations>
</target>

As you see, it’s rather complex and relies on several p2 repositories. The last repository is the Orbit repository; although it does not list any installable units, that is still required to resolve dependencies of Epsilon (see the last but one location). We have to consider this when defining our mirroring strategy.

As usual, we define a few properties at the beginning of the POM for specifying the versions of the plugin and the parts of the p2 update site we will mirror from:

<properties>
    <tycho-version>2.7.1</tycho-version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <eclipse-version>2022-03</eclipse-version>
    <xtext-version>2.26.0</xtext-version>
  </properties>

Let’s configure the Tycho plugin for mirroring (see the documentation of the plugin for all the details of the configuration):

<plugin>
  <groupId>org.eclipse.tycho.extras</groupId>
  <artifactId>tycho-p2-extras-plugin</artifactId>
  <version>${tycho-version}</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>mirror</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <source>
      <repository>
        <id>${eclipse-version}</id>
        <layout>p2</layout>
        <url>https://download.eclipse.org/releases/${eclipse-version}</url>
      </repository>
      <repository>
        <id>${xtext-version}</id>
        <layout>p2</layout>
        <url>https://download.eclipse.org/modeling/tmf/xtext/updates/releases/${xtext-version}</url>
      </repository>
      <repository>
        <id>itemis</id>
        <layout>p2</layout>
        <url>https://download.itemis.com/updates/releases/2.1.1/</url>
      </repository>
      <repository>
        <id>Epsilon</id>
        <layout>p2</layout>
        <url>https://download.eclipse.org/epsilon/updates/2.4/</url>
      </repository>
      <repository>
        <id>Orbit ${eclipse-version}</id>
        <layout>p2</layout>
        <url>https://download.eclipse.org/tools/orbit/downloads/${eclipse-version}</url>
      </repository>
    </source>

    <!-- List of IUs to mirror. If omitted, allIUs will be mirrored. -->
    <!-- Omitted IU version element means latest version of the IU -->
    <ius>
      <iu>
        <id>org.eclipse.equinox.executable.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.sdk.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.swtbot.eclipse.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.swtbot.ide.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.xtext.sdk.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.emf.sdk.feature.group</id>
      </iu>
      <iu>
        <id>de.itemis.xtext.antlr.feature.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.epsilon.picto.feature.feature.group</id>
      </iu>
      <iu>
        <id>org.eclipse.epsilon.emf.feature.feature.group</id>
      </iu>
    </ius>
    <!-- The destination directory to mirror to. -->
    <destination>${user.home}/eclipse-mirrors</destination>
    <includePacked>false</includePacked>
    <includeOptional>false</includeOptional>
    <!--
      because some features, like epsilon, require an old version of org.antlr.runtime:
      org.eclipse.epsilon.common 2.4.0.202203041826
      requires 'osgi.bundle; org.antlr.runtime [3.1.1,3.5.3)'
    -->
    <latestVersionOnly>false</latestVersionOnly>
  </configuration>
</plugin>

The mirror will be generated in the user home subdirectory “eclipse-mirrors” (<destination> tag); we also define a few other mirroring options. Note that in this example, we cannot mirror only the latest versions of bundles (<latestVersionOnly>), as detailed in the comment in the POM. We also avoid mirroring the entire contents of the update sites (it would be too much). That’s why we specify single installable units. Remember that also dependencies of the listed installable units will be mirrored, so it is enough to list the main ones. You might note differences between the installable units specified in the target platform definition and those listed in the plugin configuration. Indeed, the target platform file could also be simplified accordingly, but I just wanted to have slight differences to experiment with.

If you write the above configuration in a POM file (a <packaging>pom</packaging> will be enough), you can already build the mirror running:

mvn package

Remember that the mirroring process will take several minutes depending on your Internet connection speed since it will have to download about 500 Mb of data.

You can verify that all the specified repositories are needed to create the mirror correctly. For example, try to remove this part from the POM:

<repository>
  <id>Orbit ${eclipse-version}</id>
  <layout>p2</layout>
  <url>https://download.eclipse.org/tools/orbit/downloads/${eclipse-version}</url>
</repository>

Try to create the mirror, and you should see this warning message because some requirements of Epsilon bundles cannot be resolved:

[WARNING] Mirror tool: Problems resolving provisioning plan.:
[Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-gfm-strikethrough 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-gfm-tables 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-heading-anchor 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-image-attributes 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-ins 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-task-list-items 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; com.atlassian.commonmark-yaml 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; net.sourceforge.plantuml 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.picto 2.3.0.202104221823
to osgi.bundle; org.apache.commons.csv 0.0.0.;
Unable to satisfy dependency from org.eclipse.epsilon.emg.engine 2.3.0.202104221823
to osgi.bundle; org.apache.commons.math3 0.0.0.]

Those requirements are found in the Orbit p2 repository, which we have just removed for testing purposes.

Unfortunately, I found no way to make the build fail in such cases, even because it’s just a warning, not an error. I guess this is a limitation of the Eclipse mirroring mechanism. However, we will now see how to verify that the mirror contains all the needed software using another mechanism.

We create a modified version of our target definition file pointing to our local mirror. To do that, we create an Ant file (create_local_target.ant):

<?xml version="1.0" encoding="ISO-8859-1"?>
<project name="CreateLocalTargetDefinitionFile" basedir="." default="create-target-file">

	<!--
		Replaces all the repository location elements of a .target file with
		a location pointing to a local file system directory where the
		mirror has to be already created
	-->
	<property name="local.target" value="local.target" />
	<property name="orig.target" location="./example.target" />
	<property name="local.mirror.path.input" location="${user.home}/eclipse-mirrors" />
	<property name="local.mirror.url.input" value="file:${local.mirror.path}" />

	<macrodef name="replace_win_slashes">
		<attribute name="property.to.process" />
		<attribute name="output.property" />
		<sequential>
			<loadresource property="@{output.property}">
				<string value="@{property.to.process}" />
				<filterchain>
					<replaceregex pattern="\\" replace="/" flags="gi" />
				</filterchain>
			</loadresource>
			<echo message="property.to.process: @{property.to.process}" />
			<echo message="output.property    : ${@{output.property}}" />
		</sequential>
	</macrodef>

	<replace_win_slashes property.to.process="${local.mirror.path.input}" output.property="local.mirror.path" />
	<replace_win_slashes property.to.process="${local.mirror.url.input}" output.property="local.mirror.url" />

	<target name="copy-target-file" description="Copy the .target definition file into the local.target">
		<echo message="local.target: ${local.target}" />
		<echo message="orig.target : ${orig.target}" />

		<copy file="${orig.target}" tofile="${local.target}" overwrite="true" verbose="true" />
	</target>


	<target name="create-target-file" depends="copy-target-file" description="Creates a .target file from the original one, pointing to a local mirror">
		<echo message="local.mirror.path: ${local.mirror.path}" />
		<echo message="local.mirror.url : ${local.mirror.url}" />

		<replaceregexp>
			<regexp pattern="target name=&quot;(\S+)&quot;(\.*)" />
			<substitution expression="target name=&quot;local&quot;\2" />
			<fileset id="path.target" dir=".">
				<include name="${local.target}" />
			</fileset>
		</replaceregexp>

		<replaceregexp byline="true">
			<regexp pattern="&lt;repository location=&quot;(\S+)&quot;(\.*)/&gt;" />
			<substitution expression="&lt;repository location=&quot;${local.mirror.url}&quot;\2/&gt;" />
			<fileset id="path.target" dir=".">
				<include name="${local.target}" />
			</fileset>
		</replaceregexp>
	</target>

</project>

Note that this also handles path separators in Windows correctly. The idea is to replace lines of the shape <repository location=”https://…”/> with <repository location=”file:/…/eclipse-mirrors”/>. This file assumes the original target file is example.target, and the modified file is generated into local.target.

Let’s call this Ant script from the POM:

<plugin>
  <artifactId>maven-antrun-plugin</artifactId>
  <!-- create the local.target -->
  <executions>
    <execution>
      <id>create-target-file</id>
      <phase>process-resources</phase>
      <configuration>
        <target>
          <ant antfile="create_local_target.ant" 
                target="create-target-file">
          </ant>
        </target>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Finally, let’s use Tycho to validate the local.target file (see the documentation of the goal):

<plugin>
  <groupId>org.eclipse.tycho.extras</groupId>
  <artifactId>target-platform-validation-plugin</artifactId>
  <version>${tycho-version}</version>
  <executions>
    <!-- Binds by default to the lifecycle phase: validate. -->
    <execution>
      <id>validate-target-platform</id>
      <goals>
        <goal>validate-target-platform</goal>
      </goals>
      <configuration>
        <targetFiles>
          <targetFile>local.target</targetFile>
        </targetFiles>
        <checkDependencies>true</checkDependencies>
        <checkProvisioning>true</checkProvisioning>
      </configuration>
    </execution>
  </executions>
</plugin>

Now, if we run:

mvn package

we build the mirror, and we create the local.target file.

Then, we can run the above goal explicitly to verify everything:

mvn target-platform-validation:validate-target-platform@validate-target-platform

If this goal also succeeds, we managed to create a local mirror that we can use in our local builds. Of course, in the parent POM of your project, you must configure the build so that you can switch to local.target instead of using your standard .target file. (You might want to look at the parent POM of my Edelta project to take some inspiration.)

Since we should not trust a test that we never saw failing (see also my TDD book let’s try to verify with the incomplete mirror that we learned to create by removing the Orbit URL. We should see that our local target platform cannot be validated:

[INFO] Validating /home/bettini/work/eclipse/tycho/tycho-mirrors/tycho-mirrors/local.target...
[ERROR] Cannot resolve target definition:
[ERROR] Software being installed: org.eclipse.epsilon.emf.feature.feature.group ...
[ERROR] Missing requirement: org.eclipse.epsilon.emg.engine ...
requires 'osgi.bundle; org.apache.commons.math3 0.0.0' but it could not be found
[ERROR] Cannot satisfy dependency: org.eclipse.epsilon.core.feature.feature.group ...
depends on: org.eclipse.equinox.p2.iu; org.eclipse.epsilon.emg.engine ...
[ERROR] Cannot satisfy dependency: org.eclipse.epsilon.emf.feature.feature.group ...
depends on: org.eclipse.equinox.p2.iu; org.eclipse.epsilon.core.feature.feature.group ...

Alternatively, let’s try to build our mirror with <latestVersionOnly>true</latestVersionOnly>, and during the validation of the target platform, we get:

Error: Cannot resolve target definition:
Error: Software being installed: org.eclipse.epsilon.picto.feature.feature.group 2.4.0.202203041826
Error: Missing requirement:
org.eclipse.epsilon.common 2.4.0.202203041826 requires
'osgi.bundle; org.antlr.runtime [3.1.1,3.5.3)' but it could not be found

In fact, we mirror only the latest version of org.antlr.runtime (4.7.2.v20200218-0804), which does not satisfy that requirement. That’s why we must use with <latestVersionOnly>false</latestVersionOnly> in this example.

For completeness, this is the full POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>io.lorenzobettini</groupId>
  <artifactId>tycho-mirrors</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <packaging>pom</packaging>

  <properties>
    <tycho-version>2.7.1</tycho-version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <eclipse-version>2022-03</eclipse-version>
    <xtext-version>2.26.0</xtext-version>
  </properties>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-scm-plugin</artifactId>
          <version>1.11.2</version>
        </plugin>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.2.0</version>
        </plugin>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>exec-maven-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-antrun-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>build-helper-maven-plugin</artifactId>
          <version>3.2.0</version>
        </plugin>
        <plugin>
          <groupId>org.eclipse.tycho.extras</groupId>
          <artifactId>tycho-eclipserun-plugin</artifactId>
          <version>${tycho-version}</version>
        </plugin>
        <plugin>
          <groupId>org.eclipse.tycho</groupId>
          <artifactId>tycho-p2-repository-plugin</artifactId>
          <version>${tycho-version}</version>
        </plugin>
        <!-- Must be called explicitly in a separate invocation
          target-platform-validation:validate-target-platform@validate-target-platform
          see also https://github.com/eclipse/tycho/issues/350 -->
        <plugin>
          <groupId>org.eclipse.tycho.extras</groupId>
          <artifactId>target-platform-validation-plugin</artifactId>
          <version>${tycho-version}</version>
          <executions>
            <!-- Binds by default to the lifecycle phase: validate. -->
            <execution>
              <id>validate-target-platform</id>
              <goals>
                <goal>validate-target-platform</goal>
              </goals>
              <configuration>
                <targetFiles>
                  <targetFile>local.target</targetFile>
                </targetFiles>
                <checkDependencies>true</checkDependencies>
                <checkProvisioning>true</checkProvisioning>
              </configuration>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>

    <plugins>
      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <!-- create the local.target -->
        <executions>
          <execution>
            <id>create-target-file</id>
            <phase>process-resources</phase>
            <configuration>
              <target>
                <ant antfile="create_local_target.ant" 
                      target="create-target-file">
                </ant>
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.eclipse.tycho.extras</groupId>
        <artifactId>tycho-p2-extras-plugin</artifactId>
        <version>${tycho-version}</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>mirror</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <source>
            <repository>
              <id>${eclipse-version}</id>
              <layout>p2</layout>
              <url>https://download.eclipse.org/releases/${eclipse-version}</url>
            </repository>
            <repository>
              <id>${xtext-version}</id>
              <layout>p2</layout>
              <url>https://download.eclipse.org/modeling/tmf/xtext/updates/releases/${xtext-version}</url>
            </repository>
            <repository>
              <id>itemis</id>
              <layout>p2</layout>
              <url>https://download.itemis.com/updates/releases/2.1.1/</url>
            </repository>
            <repository>
              <id>Epsilon</id>
              <layout>p2</layout>
              <url>https://download.eclipse.org/epsilon/updates/2.4/</url>
            </repository>
            <repository>
              <id>Orbit ${eclipse-version}</id>
              <layout>p2</layout>
              <url>https://download.eclipse.org/tools/orbit/downloads/${eclipse-version}</url>
            </repository>
          </source>

          <!-- List of IUs to mirror. If omitted, allIUs will be mirrored. -->
          <!-- Omitted IU version element means latest version of the IU -->
          <ius>
            <iu>
              <id>org.eclipse.equinox.executable.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.sdk.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.swtbot.eclipse.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.swtbot.ide.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.xtext.sdk.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.emf.sdk.feature.group</id>
            </iu>
            <iu>
              <id>de.itemis.xtext.antlr.feature.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.epsilon.picto.feature.feature.group</id>
            </iu>
            <iu>
              <id>org.eclipse.epsilon.emf.feature.feature.group</id>
            </iu>
          </ius>
          <!-- The destination directory to mirror to. -->
          <destination>${user.home}/eclipse-mirrors</destination>
          <includePacked>false</includePacked>
          <includeOptional>false</includeOptional>
          <!--
            because some features, like epsilon, require an old version of org.antlr.runtime:
            org.eclipse.epsilon.common 2.4.0.202203041826
            requires 'osgi.bundle; org.antlr.runtime [3.1.1,3.5.3)'
          -->
          <latestVersionOnly>false</latestVersionOnly>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

And this is the YAML file to build and verify in GitHub Actions:

name: Mirror and Verify

on:
  push:
    paths-ignore:
      - 'README.md'
  pull_request:
    paths-ignore:
      - 'README.md'

jobs:
  build:
    strategy:
      matrix:
        os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
      fail-fast: false
    runs-on: ${{ matrix.os }}

    steps:
    - uses: actions/checkout@v2
    - name: Set up JDK 11
      uses: actions/setup-java@v1
      with:
        java-version: 11
    - name: Cache Maven packages
      uses: actions/cache@v2
      with:
        path: ~/.m2
        key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml', '**/*.yml', '**/*.target') }}
        restore-keys: ${{ runner.os }}-m2-
    - name: Cache Mirror
      uses: actions/cache@v2
      with:
        path: ~/eclipse-mirrors
        key: ${{ runner.os }}-p2-mirror-${{ hashFiles('**/pom.xml', '**/*.target') }}
        # the key match must be perfect:
        # if we change the mirror or the tp we invalidate the cache
    - name: Build the Mirror
      run: mvn package
      working-directory: tycho-mirrors
    - name: Verify the TP against the Mirror
      run: mvn target-platform-validation:validate-target-platform@validate-target-platform
      working-directory: tycho-mirrors
    - name: Show output contents
      run: tree ~/eclipse-mirrors
      if: runner.os == 'Linux'

I hope you found this post valuable, and happy mirroring!

Discover a safer alternative to mapping keys to multiple values in Java.

I need a Map with multiple values

At some point in your years of coding in Java, you have probably needed a Map<K, Collection<V>>. If you haven’t needed one yet, give it some time… you will. There are more convenient ways to manage the Collection part of the Map using methods available since Java 8. If you have never used computeIfAbsent before, it is a method well worth learning. The following is an example of creating a Map<String, List<String>> using computeIfAbsent.

The way computeIfAbsent works is simple. First, it looks in the map for a specified key (e.g. “A”). If it doesn’t find a value at the key, it evaluates the specified Function, and stores the result in the Map. It then returns either the value it found or the value it created and stored in the Map. So basically, computeIfAbsent is a get with a put where the value put into the Map is returned by the method. This method will guarantee a value always exists for any key looked up, but can be wasteful as a replacement for get, since it will always result in a put in the cases where you are looking up a key doesn’t exist.

Map has a null problem

One issue with using Map is that implementations like HashMap allow null keys, values and returns. If we ignore null keys for a second, the problem with null values and returns is that if you look up a key that does not exist in the Map, you will get a null back. You can try and protect your code against the possibility of checking the Map returns a null when calling get or by using a safer method like getOrDefault which could return an empty collection in the default case of a multi-valued Map. The real solution would be to create a Map type that knows that a missing key should result in an empty Collection being returned when calling get. This is what a Multimap does.

Don’t clown around with Maps w/ multi-values

If you need multi-valued Map support, then consider using a Multimap type from a library like Eclipse Collections, Google Guava or Apache Commons Collections. The method put on a Multimap will know that the values are multi-valued and should automatically call add on the value containers. The method get on a Multimap knows when a key is not contained in the Multimap, and that an empty Collection should be returned instead. The point is that a Multimap has more intimate knowledge about the value type it manages. A Map may be provided the value type via generics, but it does not know that the value type must be a type of Collection.

Using a Multimap in Eclipse Collections

The following code shows the equivalent solution to the Map code above using a Multimap type from Eclipse Collections.

First, I create a specific type of Multimap, which in this case is a MutableListMultimap. Then I can simply call put with each key and value. The Multimap knows to create a backing Collection container (in this case a MutableList) for each new key. Finally, the call to multimap.get(“D”) returns an empty List. This List is not stored in the Multimap, so the Multimap will remain sparse and only contain the keys with actual values.

Multimap types in Eclipse Collections

There are several concrete Multimap types in Eclipse Collections. There are Readable, Mutable and Immutable interfaces. Multimap is the parent interface. MutableListMultimap is a leaf interface. The following are examples of the basic concrete Multimap types that can be created by using the Multimaps factory.

With Multimap, there are specific types based on the value containers (e.g. List, Set, Bag). There are specializations and sometimes optimizations that may exist for those types.

GroupBy should return a Multimap

Eclipse Collections is the only Java library that I am aware of today that returns a Multimap from its groupBy methods on each of its basic Collection types. Each specific type like MutableList or MutableSet, will return the appropriate Multimap type in its groupBy method based on its type. MutableList returns a MutableListMultimap from its groupBy method, and MutableSet returns a MutableSetMultimap. The following is an example of using groupBy in Eclipse Collections.

For more examples of groupBy, you can refer to the following blog.

EC by Example: GroupBy

More information on Eclipse Collections Multimap

Nikhil Nanivadekar wrote a blog on Multimaps in Eclipse Collections a few years ago. He gives more in depth explanations on the implementations.

Multimap — How it works

Summary

Do you need a Map with support for keys and multiple values? Consider using a Multimap from Eclipse Collections. If you’re already using other Collection types from Eclipse Collections, the groupBy methods will give you easy access to creating appropriate Multimap types.

I am a Project Lead and Committer for the Eclipse Collections OSS project at the Eclipse Foundation. Eclipse Collections is open for contributions. If you like the library, you can let us know by starring it on GitHub.

Other Java articles you may like

• The 2022 Java Programmer RoadMap
• Top 22 Java Libraries and APIs Every Programmer Should Learn in 2022
• Top 10 Spring MVC + REST Interview Questions Answers for Beginners and Experienced Developers

Map vs. Multimap was originally published in Javarevisited on Medium, where people are continuing the conversation by highlighting and responding to this story.

For over 20 years the Eclipse IDE platform, along with the Eclipse Rich Client Platform (RCP), have provided core technologies for building richly featured language IDEs, products, and applications that are portable across Windows, Mac, and Linux desktops. However, time moves on and the next generation of desktop products and applications are now being built with web technologies. In many scenarios there is a need to support both desktop and web deployments with the same functionality, and obviously those who have this requirement would ideally like to support it using a single platform. 

With this shift towards web and cloud development, many Eclipse platform adopters are now evaluating how to best migrate their existing tools, IDEs and applications. One technology to consider is Eclipse Theia. Theia is a platform that can be used for building both web and desktop IDEs and tools, based on modern, state-of-the-art web technologies (TypeScript, CSS, HTML). This often leads to the question: Is Eclipse Theia the next generation of Eclipse?

EclipseSource, a member of the Eclipse Cloud DevTools Working Group,  recently published a blog post asking this question. The article discusses requirements for a tool platform and how both Eclipse desktop and Eclipse Theia address these requirements. Ultimately, they come to the conclusion that Eclipse Theia can indeed be considered the next generation platform for building portable applications. And I agree. Eclipse Theia is indeed the next generation tooling and applications platform from the Eclipse Foundation!

Just to be clear, this is not an announcement of the deprecation of the Eclipse IDE, the Eclipse Tool Platform or Eclipse RCP. These projects are stable, widely used, well maintained, and will continue to be so for a long time. The timeframe of course depends on the health and activity of the ecosystem and the community, which is now the focus of the Eclipse IDE Working Group created last year to ensure the long-term sustainability of the Eclipse IDE and Platform. I highly recommend any company building products or critical business applications on the Eclipse platform to join that group. At the same time, we are clearly seeing a shift of developer tools and IDEs towards web-based technology, and ultimately the cloud. As a result, many projects currently based on Eclipse desktop technologies are asking what comes next.

The Eclipse ecosystem has always combined sustainability, innovation, and vendor neutral collaboration. For the last 20 years, the Eclipse desktop ecosystem has been an exemplar of this, and it will continue to be a focus of the Foundation. At the same time, we continue to innovate, e.g. with Eclipse Theia and other related technologies such as Eclipse Che, Eclipse GLSP, and EMF.cloud. This is the beauty of an industry-driven open source ecosystem like Eclipse. It addresses the requirements of adopters to have a stable platform, while also providing paths to move forward and innovate.

Despite not sharing a single line of code, in many ways Theia is an evolution of the Eclipse Tools Platform. Theia builds on wisdom distilled from two decades of engineering at Eclipse, in order to inspire the next generation. Besides the obvious benefit of simply offering a web-based technology stack, Theia is slimmer, and able to lean more heavily on aspects of the web technology stack. It does not, for example, provide its own UI technology (as Eclipse needed to do with SWT). It also doesn’t provide a new module system (as Eclipse did with OSGi). Instead, it is based on available technologies such as HTML/TypeScript, Node, VS Code extensions, and the Monaco Code Editor. This is great for the sustainability of the project. By maintaining less code and reusing more standard technologies, development resources can be focused more on the core capabilities of the platform.

Theia also has a very healthy community of active contributors, adopters and funding organizations. It is seeing widespread and mainstream adoption, serving as the platform for notable commercial technologies, including the Arduino IDE, Arm’s mbed studio, and the Google Cloud Shell Editor. There is also a wealth of extensions freely available for Theia at the Open VSX Registry. 

Theia on openHub

I should also point out that along with Theia, there are several additional technologies that help create a solid ecosystem for the next generation tool platform at the Eclipse Foundation. To mention just a few, Eclipse Che offers online workspace management; Eclipse GLSP provides support for building diagram editors in the browser; Eclipse CDT.cloud for building customizable web-based C/C++ tools and EMF.cloud moves the Eclipse modeling ecosystem to the web.

We are very happy to see Theia flourishing and the robustness of its community.  Theia certainly is the central building block of the new generation of tools that want to benefit from web-based technologies and cloud deployments. And so, yes, in this context, Theia and its ecosystem can be considered the next generation of Eclipse Platform.

2022-04-19: Edited to update the contributors logo graphic

We planted the seed of the new Eclipse Sirius Web and OCP’s 2022.3.0 release a few weeks ago and now the first buds appear! You are at the right place to discover the new Sirius capabilities (and poor you… the new fresh soundtrack coming along!).

Components diagrams

With Sirius Web, we aim to provide a smooth user experience, to smoothly edit, move, reconnect and layout diagrams. During this iteration, the team focused its efforts on removing all the obstacles to create and support component-like diagrams with ports.

This release improves lots of small things to make it possible to create such diagrams:

• The label of an edge can be displayed.

• Ports, aka bordered nodes, were enhanced during the iteration, it is now possible to:

• Snap: The user can move the border node only to the side of its parent node. The border node enters its parent node with 8px. The ELK automatic layout was adapted to have the same behavior as the manual layout.
• Display the label: The border nodes’ label is visible out of the node.
• Resize: The resizing of the border nodes was improved.

• Node border support was improved:

  • Image nodes: Nodes which use images can now have a border with all the relevant properties: color, size, radius, and line style. This applies to modelers using the compatibility layer and the web-based diagram definitions.
  • Web definition : It is now possible to configure all properties of a node’s border in web-based diagrams, including the border line style.

• Explorer:

  • It is now possible to expand or collapse items in the explorer without selecting them by clicking directly on the expand/collapse arrow icon.
  • When selecting an element or opening a representation (for example from its URL or from the onboard area), it is automatically made visible and selected in the explorer.

• Select representation opened from a tab.
• Multiline labels are supported.

Notice that some more improvements will come in the next release mostly around bendpoints’ support to better manage edges’ layout.

Tables!

The laying of the first foundation stone is done. We introduced a new kind of representation to represent tables. For instance, this kind of table can be defined programmatically in order to display the childs of a model element.

By default, this table component comes with:

• alphabetical column ordering,
• synchronization with Explorer/Details views,
• pagination,
• filter content,
• multiple types of widgets for edition: text fields, lists, multi select lists…

Some additional features will soon be ready and we are cooking something really cool for the next release about tables! Stay tuned!

That comes in spring ��

Spring is the season of love, love is around! And you know that love is not only great demonstrations but above all, lots of small attentions… hence we gave our love to many little things in Sirius Web during this iteration:

💓 Form

• The form representation now supports multiple elements as an input which allows to implement multi selection features as below we can select multiple elements in the explorer which are visible in the details view as different tabs.

• Support for displaying details on arbitrary element kinds. This is the first step for instance to implement such feature existing in Sirius Desktop: select a diagram in the explorer to get its properties displayed in the details view:

• Support for links in the form representation.

� Compatibility

We improved as well the compatibility layer which allows to reuse Sirius Desktop specification files aka odesign. We:

• Provide a variable to detect which environment is used. The value of the variable will change to something specific for the integrating application (for example siriusWeb). This is only available for the diagram for now.
• Add support for the Navigation model operation from Sirius Desktop.
• Add support for OperationAction. The actions are converted to regular tools available in the palette of the frontend.

💕 Tools

The tools were enriched as we added support for:

• tools preconditions,
• and post selection.

� Export SVG

Finally, we added the ability to export diagrams as SVG images.

Reduce, Reuse, Recycle!

Sirius Web comes with a default application but above all it relies on many reusable components. As I explained in my previous post, Sirius Web depends on the Sirius Components. For a clearer status about what is part of these reusable components and what is part of the default sample application, we renamed the namespaces of the projects from sirius-web-xxx to sirius-components-xxx. The projects sirius-web-api and sirius-web-core-api have been merged into sirius-components-core since there was no difference in scope between both projects and neither of them was strictly limited to an API. Since we have validated that most of our projects can be reused outside of a Spring environment (i.e. an environment where Spring Frameworks is in charge of the ApplicationContext), we have removed the -spring- part from most of our project names.

Other enhancements developed by the team during this iteration were around the workbench. We went one step further for the one who wants to reuse the Sirius Web workbench to build their own application. The integration of the details view in the workbench is not limited to semantic objects anymore. Now, any object can be used as the input of the details view and we provide the identifier of all the objects in the selection. This may include graphical elements such as nodes, edges, representations or anything selected in the explorer for example.

Moreover, the workbench no longer hard-codes the views which are visible in the left and right side-panels. Instead, each application must explicitly configure its workbench using the new WorkbenchViewContribution element. The team uses the Architecture Decision Records to document all the decisions made during the project life by the team. For example, for this one you can see ADR-046 for more details.

Next

To conclude, what’s on the table for the next release?

• Easy Peasy Tables definition !
• User experience continuous effort, mainly focused on the diagrams and the edge concept by finishing the work on the bendpoints’ integration, by allowing reconnection…

As usual, a detailed release notes for 2022.3.0 is available on our online documentation: http://docs.obeostudio.com.

Again, all these changes are made available thanks to our precious’ customers sponsoring the Sirius Web open source project! Do not hesitate to join the party and become a Sirius Web supporter, it is as easy as sending me an email, or contact the team.

See you soon for the next Sirius Web and OCP release!

We are happy to announce the Eclipse Theia 1.24 release! This release contains an amazing number of 78 merged pull requests...

The post Eclipse Theia 1.24 Release: News and Noteworthy appeared first on EclipseSource.

The Eclipse Ditto teams announces availability of Eclipse Ditto 2.4.0.

The main topics in this release were the move from Java 11 to Java 17 (and the switch in Ditto’s pre-built Docker containers from OpenJ9 to Hotspot runtime), W3C WoT (Web of Things) integration and enhanced placeholder capabilities, e.g. used in JWT claim extraction and signal enrichment.

Adoption

Companies are willing to show their adoption of Eclipse Ditto publicly: https://iot.eclipse.org/adopters/?#iot.ditto

From our various feedback channels we however know of more adoption.
If you are making use of Eclipse Ditto, it would be great to show this by adding your company name to that list of known adopters.
In the end, that’s one main way of measuring the success of the project.

Changelog

The main improvements and additions of Ditto 2.4.0 are:

• W3C WoT (Web of Things) integration
• SSE (ServerSentEvent) API for subscribing to messages
• Recovery status for connections indicating when e.g. recovery is no longer tried after max backoff
• Enhance placeholders to resolve to multiple values
• Advanced JWT placeholder operations
• Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition

The following notable fixes are included:

• Several fixes and improvements regarding consistency and performance of search updates
• Don’t publish messages with failed enrichments and issue failed ack
• Filter for incorrect element types in jsonArray of feature definitions
• Fix of placeholder resolvment in “commandHeaders” of “ImplicitThingCreation” mapper
• Fix fn:substring-after() function returning incorrect data

The following non-functional work is also included:

• Upgrade of compiler target level for service modules from Java 11 to Java 17
• Switch of used Java runtime in pre-built Docker containers from OpenJ9 to Hotspot
• Publication of pre-built multi-architecture Docker images for linux/amd64 (as always) and now in addition linux/arm64
• Removal of rate limiting / throttling limits as default
• Update of several used dependencies

Please have a look at the 2.4.0 release notes for a more detailed information on the release.

Artifacts

The new Java artifacts have been published at the Eclipse Maven repository as well as Maven central.

The Ditto JavaScript client release was published on npmjs.com:

• @eclipse-ditto/ditto-javascript-client-dom
• @eclipse-ditto/ditto-javascript-client-node

The Docker images have been pushed to Docker Hub:

• eclipse/ditto-policies
• eclipse/ditto-things
• eclipse/ditto-things-search
• eclipse/ditto-gateway
• eclipse/ditto-connectivity
• eclipse/ditto-concierge

–
The Eclipse Ditto team

Xtext is very good at inferring a metamodel from a grammar but it has its limitations. After reaching a certain complexity, I think it is important to decouple the grammar from the metamodel. This is allows to separate concerns: the metamodel becomes the “business model layer”, and the grammar becomes the “presentation layer”. To do this, we need to transform our project to this new setup. In this post I explain how. The full code for this example can be found here.

Overview

There are several steps to perform this migration:

• Move the generated model from the model/generated/ folder.
• Update the location of the .ecore and .genmodel file in the files that reference them.
• Update the grammar (.xtext file) so that the model is no longer generated.
• Modify the MWE2 workflow to process automatically generate the model.

Move the generated model

Xtext generates an .ecore and a .genmodel file from the grammar at folder model/generated/. Since the file is no longer generated, it is a good idea to move it to another folder, for example, the model/folder. It is important remove the original .ecore and .genmodel files, since there might be strange errors in the editors if there are two .ecore files with the same identifiers.

Update the location

There are several files in the project that reference the files in the model/generated folder. These are:

• build.properties, where we need to update the path in the bin.includes section.
• plugin.xml, where we need to update the path to the genmodel file.

Update the grammar

If you are using a generated ecore file, then you have a line like this in your .xtext file:

generate block "http://www.idiomaticsoft.com/dsl/block/Block"

This line needs to be replaced by the following line:

import "http://www.idiomaticsoft.com/dsl/block/Block"

Thus indicating Xtext that it needs to import the metamodel with that identifier.

Modify the MWE2 workflow

The mwe2 workflow is heavily modified. We start from the the mwe2 file of the In a block language and transform it as follows:

module com.idiomaticsoft.dsl.block.GenerateBlock

import org.eclipse.xtext.xtext.generator.*
import org.eclipse.xtext.xtext.generator.model.project.*
import org.eclipse.emf.mwe.utils.*

var rootPath = ".."
var projectName = "com.idiomaticsoft.dsl.block"
var runtimeProject = "../${projectName}"

Workflow {
	// This cleans the directory before generating
	component = DirectoryCleaner {
		directory = "${runtimeProject}/src-gen"
	}
	// This handles the generation of the model classes
	component = org.eclipse.emf.mwe2.ecore.EcoreGenerator {
		genModel = "platform:/resource/com.idiomaticsoft.dsl.block/model/Block.genmodel"
		srcPath = "platform:/resource/com.idiomaticsoft.dsl.block/src-gen"
	}
	  // This handles the generation of the standalone setub
    bean = org.eclipse.emf.mwe.utils.StandaloneSetup {
        scanClassPath = true
        platformUri = "${rootPath}"
     }	

	component = XtextGenerator {
    // We tell the xtext generator to avoid cleaning, because otherwise it
    // would remove the classes generated by the ecore
		cleaner = {
			enabled = false
		} 
		configuration = {
			project = StandardProjectConfig {
				baseName = "com.idiomaticsoft.dsl.block"
				rootPath = rootPath
				runtimeTest = {
					enabled = true
				}
				eclipsePlugin = {
					enabled = true
				}
				eclipsePluginTest = {
					enabled = true
				}
				createEclipseMetaData = true
			}
			code = {
				encoding = "UTF-8"
				lineDelimiter = "\n"
				fileHeader = "/*\n * generated by Xtext \${version}\n */"
				preferXtendStubs = false
			}
		}
		language = StandardLanguage {
			name = "com.idiomaticsoft.dsl.block.Block"
			fileExtensions = "block"
      // we tell the generator that we are using our Block.genmodel file
			referencedResource = "platform:/resource/com.idiomaticsoft.dsl.block/model/Block.genmodel"
			serializer = {
				generateStub = false
			}
			validator = {
				composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
				// Generates checks for @Deprecated grammar annotations, an IssueProvider and a corresponding PropertyPage
				generateDeprecationValidation = true
			}
			generator = {
				generateXtendStub = true
			}
			junitSupport = {
				junitVersion = "5"
			}
		}
	}
}

Conclusion

After all those modifications, we can modifying our .ecore files separately and they will be integrated with our Xtext grammar.