Amazon Managed Service for Apache Flink

On August 30, 2023, AWS renamed Amazon Kinesis Data Analytics to Amazon Managed Service for Apache Flink. The new name better reflects that the service uses Apache Flink under the hood.

What Changed

The name change clarified what the service does. Amazon Kinesis Data Analytics was a mouthful that didn’t tell you much. Amazon Managed Service for Apache Flink is more honest: it’s a managed service for running Apache Flink applications.

AWS made the transition smooth. Service endpoints, APIs, the AWS CLI, IAM policies, CloudWatch metrics, and the Billing console all stayed the same. Your existing applications kept working without modification.

What the Service Does

Amazon Managed Service for Apache Flink lets you build and run Apache Flink applications without managing the infrastructure. The service handles scaling, patching, and maintenance. You write your application code; AWS handles the rest.

The service can run tens of thousands of Apache Flink applications simultaneously. This matters if you’re processing real-time data at scale.

Capabilities

The service gives you access to Apache Flink’s core features:

• Low-latency processing: Sub-second data processing for time-sensitive applications
• High-throughput handling: Can process millions of events per second
• Exactly-once processing semantics: Each event processes exactly once, with no duplicates
• Durable state management: Application state persists across failures

Integrations

The service connects with other AWS services:

• Amazon MSK (Managed Streaming for Apache Kafka)
• Amazon Kinesis Data Streams
• Amazon OpenSearch Service
• Amazon DynamoDB Streams
• Amazon S3

You can also build custom integrations with other services.

Security

The service encrypts data in transit and at rest. It integrates with IAM for access control. The underlying infrastructure runs across multiple availability zones for high availability.

Region Availability

Amazon Managed Service for Apache Flink is available in most AWS Regions. Check the AWS Region table for current availability.

Getting Started

To start using the service, open the AWS Management Console and search for Amazon Managed Service for Apache Flink. The documentation includes getting started guides and API references.

FAQ

Q: What is Amazon Managed Service for Apache Flink?

A: It’s a managed service for Apache Flink. You can build applications that process streaming data in real time without managing the underlying infrastructure.

Q: How does the rename from Amazon Kinesis Data Analytics affect me?

A: The rename was mostly cosmetic. Console labels and documentation changed, but the APIs, CLI commands, and endpoints stayed the same. Your existing applications continued working.

Q: What can I do with this service?

A: Common use cases include real-time analytics, ETL pipelines, and event-driven applications. The service processes data from Kinesis Data Streams, Amazon MSK, DynamoDB Streams, and other sources.

Q: How does it handle failures?

A: Apache Flink checkpoints application state periodically. If a failure occurs, the application resumes from the last checkpoint without data loss or duplicates.

Q: Which AWS services does it integrate with?

A: Built-in connectors exist for Amazon MSK, Kinesis Data Streams, OpenSearch Service, DynamoDB Streams, and S3. You can also create custom connectors.

Q: Is the service available globally?

A: It’s available in most AWS Regions. Some regions may have limited or no availability. Check the AWS Region table for specifics.