The new architecture allows flexible development, deployment and operations of business and integration services. Besides, it is vendor-agnostic so that you can leverage on-premise hardware, different public cloud infrastructures, and cloud-native PaaS platforms.

The session will describe the challenges of the existing monolith system, the step-by-step procedure to move to the new cloud-native Microservices architecture. It also explains why containers such as Docker play a key role in this scenario.

A live demo shows how container solutions such as Docker, PaaS cloud platforms such as CloudFoundry, cluster managers such as Kubernetes or Mesos, and different programming languages are used to implement, deploy and scale cloud-native Microservices in a vendor-agnostic way.

Key Takeaways

Key takeaways for the audience:

– Best practices for moving to a cloud-native architecture

– How to leverage microservices and containers for flexible development, deployment and operations

– How to solve challenges in real world projects

– Understand key technologies, which are recommended

– How to stay vendor-agnostic

– See a live demo of how cloud-native applications respectively services differ from monolith applications regarding development and runtime

Slides and Video from Microservices Meetup Mumbai

Here are the slides and video recording. Presented in February 2017 at Microservices Meetup Mumbai, India.

The post Case Study: From a Monolith to Cloud, Containers, Microservices appeared first on Kai Waehner.

I had three talks:

• How to Apply Machine Learning to Real Time Processing
• Comparison of Open Source IoT Integration Frameworks
• Tools in Action – Live Demo of Open Source Project Flogo

In addition, I was interviewed by the Voxxed team about Big Data, Machine Learning and Internet of Things. The video will be posted on Voxxed website in the next weeks.

You can find several slides and video recordings about my big data / machine learning topic on my blog / website. Therefore, I will focus on the open source IoT frameworks in this post.

Open Source Integration Frameworks for the Internet of Things

Internet of Things (IoT) and edge integration are getting more important than ever before due to the massively growing number of connected devices year by year. In this talk, I showed open source frameworks built to develop very lightweight microservices. These can be deployed on small devices or in cloud native containers / serverless architectures with very low resource consumption to wire together all different kinds of hardware devices, APIs and online services.

The focus of this session was to discuss open source process engines such as Eclipse Kura (in conjunction with Apache Camel), Node-RED or Flogo. These offer a framework plus zero-code environment with Web IDE for building and deploying IoT integration and data processing directly onto IoT gateways and connected devices. For that, they leverage IoT standards such as MQTT, WebSockets or CoaP, but also other interfaces such as Twitter feeds or REST services.

The session also discussed the relation to other components in a IoT architecture including:

• Open source dataflow pipelines (like Apache NiFi / MiNiFy, StreamSets or Cask Hydrator),
• Open source stream processing frameworks (such as Apache Storm, Flink, Spark Streaming or Samza)
• Cloud IoT platforms (such as AWS IoT, Cloud IoT Cloud Platform or IBM’s open source serverless framework OpenWhisk).

Slide Deck: Apache Nifi vs. StreamSets vs. Eclipse Kura vs. Node-RED vs. Flogo vs. Others

Here is the slide deck about different alternatives for IoT integration:

Video Recording: Integration Frameworks for the Internet of Things

And here is the video recording where I walk through the above slide deck and also show live demos of Node-Red and Flogo:

As always, I appreciate any comments or feedback…

The post Comparison of Open Source IoT Integration Frameworks appeared first on Kai Waehner.

What is Project Flogo?

In short, Flogo is an ultra-lightweight integration framework powered by Go programming language. It is open source under the permissive BSD license and easily extendable for your own use cases. Flogo is used to develop IoT edge apps or cloud-native / serverless microservices. Therefore, it is complementary to other integration solutions and IoT cloud platforms.

Some key characteristics:

• Ultra-light footprint (powered by Golang) for edge devices with zero dependency model, very low disk and memory footprint, and very fast startup time
• Can be run on a variety of platforms (edge device, edge gateway, on premise, cloud, container)
• Connectivity to IoT technologies (MQTT, CoaP, REST, …)
• Highly optimized for unreliable IoT environments
• Intended to be used by developers / integration specialists / citizen integrators either by writing source code or leveraging the Web UI for visual coding, testing and debugging
• Includes some innovating features like a web-native step-back debugger to interactively design / debug your process, simulate sensor events, and change data / configuration without restarting the complete process

Overview, Architecture and Use Cases

The following slide deck shows an overview, architecture and use cases for Flogo:

You can also watch the following 45min video where I walk you through these slides and also show some live demos and source code:

Flogo Live Demo and Source Code

If you just want to see the live demo, watch the following 15min video:

Any feedback or questions are highly appreciated. Please use the Community Q&A for to ask whatever you want to know.

The post Open Source Project Flogo – Overview, Architecture and Live Demo appeared first on Kai Waehner.

Enterprise Integration Patterns

The post Enterprise Integration Patterns (EIP) Revisited in 2014 appeared first on Kai Waehner.

Content

Applications have to be integrated – no matter which programming languages, databases or infrastructures are used. However, the realization of integration scenarios is a complex and time-consuming task. Over 10 years ago, Enteprise Integration Patterns (EIP) became the world wide defacto standard for splitting huge, complex integration scenarios into smaller recurring problems. These patterns appear in almost every integration project.
This session revisits EIPs and gives shows status quo. After giving a short introduction with several examples, the audience will learn which EIPs still have a „right to exist“, and which new EIPs emerged in the meantime. The end of the session shows different frameworks and tools which already implement EIPs and therefore help the architect to reduce efforts a lot.

Slides

The post Slides online: “Enterprise Integration Patterns Revisited” – Talk at OBJEKTspektrum Information Days 2013 appeared first on Kai Waehner.

Abstract

Big data represents a significant paradigm shift in enterprise technology. Big data radically changes the nature of the data management profession as it introduces new concerns about the volume, velocity and variety of corporate data. Apache Hadoop is the open source defacto standard for implementing big data solutions on the Java platform. Hadoop consists of its kernel, MapReduce, and the Hadoop Distributed Filesystem (HDFS). A challenging task is to send all data to Hadoop for processing and storage (and then get it back to your application later), because in practice data comes from many different applications (SAP, Salesforce, Siebel, etc.) and databases (File, SQL, NoSQL), uses different technologies and concepts for communication (e.g. HTTP, FTP, RMI, JMS), and consists of different data formats using CSV, XML, binary data, or other alternatives. This session shows different open source frameworks and products to solve this challenging task. Learn how to use every thinkable data with Hadoop – without plenty of complex or redundant boilerplate code.

Slides

The post WJAX 2013 Slides online: Big Data beyond Apache Hadoop – How to integrate ALL your Data with Camel and Talend appeared first on Kai Waehner.

Abstract

SQL cannot solve several problems emerging with big data. A distributed, fault-tolerant architecture is necessary. NoSQL comes to the rescue, but therefore it does not use SQL as its query language or give full ACID guarantees. Thus, in the future you will have to learn new concepts and integrate these NoSQL databases as you integrate SQL databasestoday. The open source integration framework Apache Camel is already prepared for this challenging task.

Apache Camel implements the well-known Enteprise Integration Patterns (EIP) and therefore offers a standardized, domain-specific language to integrate applications and clouds. It can be used in almost every integration project within the JVM environment. All integration projects can be realized in a consistent way without redundant boilerplate code.

This session demonstrates the elegance of Apache Camel for NoSQL integration. Several examples are shown for all different concepts by integrating NoSQL databases from CouchDB (Document Store), HBase (Column-oriented), Neo4j (Graph), Amazon Web Services (Key Value Store), and others.

If the required NoSQL database is not supported by Apache Camel, you can easily create your own Camel component with very low effort. This procedure is explained at the end of the session.

Slides

The post JBoss OneDayTalk 2013: “NoSQL Integration with Apache Camel – MongoDB, CouchDB, Neo4j, Cassandra, HBase, Hazelcast, Riak, etc.” appeared first on Kai Waehner.

Apache Camel is the best integration framework “on the market”. It has very good domain specific languages, many connectors, different companies behind it, and an awesome worldwide open source community. So, seeing a new book about Apache Camel is always good news!

Danger of Confusion

There are two new Apache Camel books:

• Apache Camel Messaging System, released in September 2013 by PACKT PUBLISHING, see this book review
• Apache Camel Message Routing, released in August 2013 by PACKT PUBLISHING, see http://www.kai-waehner.de/blog/2013/10/04/book-review-apache-camel-message-routing/

Sorry PACKT, this is ridiculous! Two books within one month with same name and almost same content. Therefore, both reviews are very similar. Why not connecting both authors to write ONE book ?! Be sure to buy just one of the two books!

Content

First of all, the book has only 70 pages. So it does not contain that much content. It offers a short introduction to Apache Camel. The book explains in detail what Camel is, how to install it, and how to get started. You will also learn about different domain specific languages and some Enteprise Integration Patterns.

Compare to the other new Camel book by PACKT, this one differs in two things:

• The introduction / theory is more detailed
• The practical examples are less detailed

Both alternatives are a good introduction to Camel, you can get started easily with both!

Conclusion

Good news:

• Yet another new Apache Camel book
• Easy to read
• Good examples
• Good starting point for newbies

Bad news:

• Just 70 pages. Just for getting started. Therefore, very expensive.
• You can find all information of this book on Camel’s website for free (though, newbies might not find all this information easily by searching the website)
• “Camel in Action” is also available. That’s another awesome Apache Camel book, which also explains all basics, but also many many more details (500 pages)
• If you already know Apache Camel, you do NOT need this book

Summary:

If you are looking an easy-to-read book for getting started with Apache Camel, then this book is for you. Afterwards, you still need to buy “Camel in action”, too. “Camel in action” is not as easy as this one for getting started, as even the first chapters contain many details. This might be to much for newbies. So, it is no bad idea to start with this book, then buy “Camel in Action” for using Apache Camel in your projects.

Best regards,

Kai Wähner

Twitter: @KaiWaehner

Website: www.kai-waehner.de

The post Yet another new Camel book: “Apache Camel Messaging Systems” (Danger of Confusion) – PACKT PUBLISHING appeared first on Kai Waehner.

Apache Camel is the best integration framework “on the market”. It has very good domain specific languages, many connectors, different companies behind it, and an awesome worldwide open source community. So, seeing a new book about Apache Camel is always good news!

Content

First of all, the book has only 50 pages. So it does not contain that much content. It offers a short introduction to Apache Camel. Afterwards, it explains several Enterprise Integration Patterns such as Splitter, Aggregator or Content-based Router, and how to implement these patterns with Apache Camel. The book uses nice screenshots and offers step-by-step explanations in each section. You can also download the source code examples for getting started quickly.

Conclusion

Good news:

• Finally, a new Apache Camel book
• Easy to read
• Good examples
• Good starting point for newbies

Bad news:

• Just 50 pages. Just for getting started. Therefore, very expensive.
• You can find all information of this book on Camel’s website for free (though, newbies might not find all this information easily by searching the website)
• “Camel in Action” is also available. That’s another awesome Apache Camel book, which also explains all basics, but also many many more details (500 pages)
• If you already know Apache Camel, you do NOT need this book

Summary:

If you are looking an easy-to-read book for getting started with Apache Camel, then this book is for you. Afterwards, you still need to buy “Camel in action”, too. “Camel in action” is not as easy as this one for getting started, as even the first chapters contain many details. This might be to much for newbies. So, it is no bad idea to start with this book, then buy “Camel in Action” for using Apache Camel in your projects.

Best regards,

Kai Wähner

Twitter: @KaiWaehner

Website: www.kai-waehner.de

The post Book Review: “Apache Camel Message Routing” appeared first on Kai Waehner.

Abstract

Data exchanges between companies increase a lot. The number of applications which must be integrated increases, too. The interfaces use different technologies, protocols and data formats. Nevertheless, the integration of these applications shall be modeled in a standardized way, realized efficiently and supported by automatic tests.

Three integration frameworks are available in the JVM environment, which fulfil these requirements: Apache Camel, Spring Integration and Mule. They implement the well-known Enteprise Integration Patterns (EIP) and therefore offers a standardized, domain-specific language to integrate applications.

These Integration Frameworks can be used in almost every integration project within the JVM environment – no matter  which technologies, transport protocols or data formats are used. All integration projects can be realized in a consistent way without redundant boilerplate code.

This session shows and compares the three alternatives and discusses their pros and cons. Besides, a recommendation will be given when to use a more powerful Enterprise Service Bus (ESB) instead of one of these frameworks.

Slides

The post How to choose the right Open Source Integration Framework – Apache Camel (JBoss, Talend), Spring Integration (Pivotal) or Mule ESB? – JavaOne 2013 appeared first on Kai Waehner.