How To Start Kafka Server In Windows
Is Windows your favorite development environment? Practice you want to run Apache Kafka® on Windows? Cheers to the Windows Subsystem for Linux 2 (WSL 2), now you can, and with fewer tears than in the by. Windows still isn't the recommended platform for running Kafka with product workloads, but for trying out Kafka, information technology works just fine. Let's take a look at how it's done.
- Gear up your surround
- Install WSL 2
- Enable the Windows Subsystem for Linux
- Enable the Virtual Machine characteristic
- Become the Linux kernel update
- Set the default WSL version
- Install your preferred Linux distribution
- Install Java
- Download Kafka
- Outset the Kafka cluster
- Produce and eat some messages
- Stop Kafka
- Install WSL 2
- Kafka and Windows: The good, the bad, and the ugly
- You're just getting started!
- Kafka on Windows? What made this possible?
Ready your environment
Install WSL 2
The Windows Subsystem for Linux ii makes it all possible. Microsoft describes WSL 2 as "a GNU/Linux environment—including most command line tools, utilities, and applications—directly on Windows, unmodified, without the overhead of a traditional virtual machine or dual kick setup."
Make certain you're running Windows 10, version 1903.18362 or higher. Click Start and navigate to Settings > Organization > About. In the "Windows specifications" section, find the "Bone build."
If you're on the Windows Update train, you probably have the latest version and are expert to go. If non, you need to update Windows 10.
When you're sure that Windows is up to appointment, follow these instructions below to install WSL 2.
Enable the Windows Subsystem for Linux
Turn on the Windows Subsystem for Linux feature before installing a Linux distribution. Open up PowerShell as an ambassador, and run the post-obit command:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
This may take a minute or ii. Your output should resemble the post-obit:
Deployment Image Servicing and Management tool Version: 10.0.18362.1139 Image Version: 10.0.18363.1139 Enabling feature(south) [==========================100.0%==========================] The operation completed successfully.
Enable the Virtual Machine feature
WSL 2 requires the Virtual Automobile Platform feature. In PowerShell, run the following command:
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
Get the Linux kernel update
Download the Linux kernel update package, which is a regular Windows Installer (.msi) file.
Double-click the .msi file to install the WSL 2 update. If you're prompted for elevated permissions, select Yeah to corroborate the installation.
Set the default WSL version
In PowerShell, run the following command to set WSL 2 as the default version for your Linux distributions:
wsl --set-default-version 2
Your output should resemble the following:
For data on central differences with WSL 2 please visit https://aka.ms/wsl2
WSL 2 is ready to utilise. For more than information on installing WSL 2, including troubleshooting, see Windows Subsystem for Linux Installation Guide for Windows x.
Install your preferred Linux distribution
Install Linux from the Microsoft Store, the aforementioned manner y'all install other applications on Windows.
Open up the Microsoft Store app and search for "Linux."
This web log postal service uses Ubuntu 20.04. Select Ubuntu twenty.04 LTS and click Install.
When the installation is complete, click Launch. The shell opens and displays the following message:
Installing, this may have a few minutes... Please create a default UNIX user account. The username does not demand to match your Windows username. For more information visit: https://aka.ms/wslusers Enter new UNIX username:
Enter a username and password to complete the installation.
Install Coffee
Run the package manager to go the latest updates. In the Ubuntu shell window that opened to a higher place, run the following commands:
sudo apt-get update && sudo apt-get upgrade -y
| ℹ️ | Tip: Right-click pastes text into the final window. |
Kafka requires the Java runtime to be version 8 or higher. Check the Java version in your Linux installation:
coffee -version
Your output should resemble this:
openjdk version "i.viii.0_265" OpenJDK Runtime Environment (build 1.8.0_265-8u265-b01-0ubuntu2~20.04-b01) OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode)
If Java isn't installed (likely) or it's not the right version, install it by using your distribution'south package manager. At that place are a lot of ways to install Java. On Ubuntu, this is one of the simplest:
sudo apt install openjdk-8-jdk -y
Download Kafka
You can install Kafka past using a package director, or you can download the tarball and extract it to your local automobile directly.
Select a mirror from the Kafka download site and download the tarball. The following command downloads Apache Kafka version 2.6:
wget https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.thirteen-ii.half-dozen.0.tgz
Run the post-obit commands to untar the Kafka archive, and cd to the kafka directory:
tar -xzf kafka_2.thirteen-2.6.0.tgz cd kafka_2.13-2.half-dozen.0
Run the ls -al command to list the contents of the kafka directory:
full 64 drwxr-xr-10 7 jim jim 4096 Oct fourteen 12:27 ./ drwxr-xr-ten 25 jim jim 4096 November 20 12:52 ../ -rw-r--r-- 1 jim jim 29975 Jul 28 11:sixteen LICENSE -rw-r--r-- 1 jim jim 337 Jul 28 11:sixteen Detect drwxr-xr-x 3 jim jim 4096 Jul 28 11:23 bin/ drwxr-xr-x two jim jim 4096 Jul 28 eleven:23 config/ drwxr-xr-x two jim jim 4096 October 14 12:26 libs/ drwxr-xr-ten 2 jim jim 4096 October fourteen 12:28 logs/ drwxr-xr-x two jim jim 4096 Jul 28 11:23 site-docs/
Outset the Kafka cluster
Run the following command to start ZooKeeper:
bin/zookeeper-server-start.sh config/zookeeper.backdrop
At that place volition be a lot of output, and ZooKeeper will exist set up in a short fourth dimension, typically around a second or two.
Open some other terminal session. Change the directory to the kafka directory, and commencement the Kafka broker:
cd kafka_2.13-2.6.0 bin/kafka-server-start.sh config/server.backdrop
If y'all adjust the windows to exist adjacent, your output should resemble the following screenshot:
ZooKeeper (left) and a Kafka broker (right) on Ubuntu 20.04 running on Windows and WSL 2
Produce and consume some messages
Open another terminal session and run the kafka-topics command to create a Kafka topic named quickstart-events:
cd kafka_2.13-2.6.0 bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
Your output should resemble this:
first issue second event third event
Arrange the producer and consumer terminal windows to be adjacent. In the producer terminal, blazon a few more messages, and watch as they appear in the consumer last.
Terminate Kafka
When you're washed experimenting with Kafka, follow these steps to leave the Kafka environs:
- End the consumer and producer clients with Ctrl+C
- Stop the Kafka broker with Ctrl+C
- Stop the ZooKeeper server with Ctrl+C
- Run the post-obit command to clean up:
rm -rf /tmp/kafka-logs /tmp/zookeeper
Kafka and Windows: The good, the bad, and the ugly
There are lots of Kafka-on-Windows tutorials, but most make the mistake of running Kafka straight on the JVM on Windows. Superficially, this appears to work, but in that location are limitations: Kafka uses specific features of POSIX to achieve high operation, and so emulations—which happen on WSL 1—are insufficient. For example, the broker will crash when information technology rolls a segment file. Ever run Kafka on Windows in a Linux environment backed by WSL 2.
Another arroyo that works well is to run Kafka in Docker containers. Docker Desktop for Windows has been updated to use the WSL 2 back end, and then Docker works exactly as it does on native Linux, without needing to spin up an unabridged VM. If you lot want to requite this approach a go, try it out using the Confluent Platform demo.
Apache Kafka and Confluent Platform running in Docker containers on Ubuntu 20.04 on Windows and WSL 2
You're just getting started!
Although Kafka provides an result streaming platform to build your applications on, you'll want to take reward of the broader ecosystem of components—similar ksqlDB, Confluent Schema Registry, and Confluent Control Middle—all provided as role of Confluent Platform.
Now that you have Kafka installed, you'll want to learn more about it, endeavour out the numerous tutorials, and join in the community! Don't forget that Apache Kafka has many APIs—including the producer and consumer but besides Kafka Streams and Kafka Connect.
Learn More
Kafka on Windows? What made this possible?
You may recall a time when Linux was anathema to Microsoft. Back in 2001, Microsoft CEO Steve Ballmer famously called Linux a "malignant cancer," just he has since come around to "loving" it. Microsoft's current CEO Satya Nadella seems intent on making it a start-course citizen in the Microsoft ecosystem, which means that a new era has arrived for software developers on the Windows platform.
When the Windows Subsystem for Linux (WSL 1) was released in 2016, it became possible to run a real Linux dev environment in a Linux shell, while retaining the familiar Windows UX around the shell. Even File Explorer was integrated nicely with the Linux file system.
The large drawbacks are that WSL ane emulates a Linux kernel, and it runs in a total VM. The first means processes that require a native kernel, like Docker, can't run. The 2nd means that WSL i consumes a lot of resources. WSL i was not sufficient to run Kafka reliably.
Just Microsoft delivered WSL 2 in 2019, and it's a whole new world. They stock-still the two biggest limitations, so WSL ii runs a real Linux kernel, and the kernel runs on a subset of Hyper-5 features, not in a full VM. For details, run across Comparison WSL 1 and WSL 2. Now the path is clear for devs to build Kafka and ksqlDB apps on Windows.
Jim Galasyn is a technical writer at Confluent, working on Kafka Streams and ksqlDB. He came to Confluent afterwards a stint at Docker, and before that, fourteen years at Microsoft writing developer documentation. Even subsequently four years of working in Silicon Valley companies, he nonetheless prefers Windows.
Source: https://www.confluent.io/blog/set-up-and-run-kafka-on-windows-linux-wsl-2/
Posted by: merrywasee1983.blogspot.com
0 Response to "How To Start Kafka Server In Windows"
Post a Comment