Download

Apache Storm 2.3.0 Released

The Apache Storm community is pleased to announce that version 2.3.0 has been released and is available from the downloads page.

This release includes a number of code improvements and important bug fixes that improve Apache Storm's performance, stability and fault tolerance. We encourage users of previous versions to upgrade to this latest release.

Thanks

Special thanks are due to all those who have contributed to Apache Storm -- whether through direct code contributions, documentation, bug reports, or helping other users on the mailing lists. Your efforts are much appreciated.

Changes in this Release

New Feature

  • [STORM-3271] - Launch storm workers in docker containers
  • [STORM-3388] - Launch workers inside container using runc runtime

Improvement

  • [STORM-3638] - Add fast and more reliable process liveness check
  • [STORM-3641] - switch JCQueue metrics to new metrics API
  • [STORM-3642] - update AutoTGT metric to new API
  • [STORM-3644] - improve PacemakerClient error messaging
  • [STORM-3647] - Add OFF-HEAP macro for worker childopts similar to LIMIT-MEM
  • [STORM-3648] - Add meter to track worker heartbeat rate
  • [STORM-3650] - Ackers and metricComponents are not distributed evenly
  • [STORM-3653] - Document treatment of common nodes in favored/unfavored nodes in scheduling
  • [STORM-3654] - JCQueue shouldn't include executor string in the metric name
  • [STORM-3656] - Change handling of Hadoop TGT renewal exception
  • [STORM-3657] - storm.messaging.netty.authentication should be true if it is true in either the topology conf or cluster conf
  • [STORM-3659] - User page should show storm Version in the Owner topologies table
  • [STORM-3660] - Remove messaging queue for updating Executor credentials
  • [STORM-3661] - Netty server should handle incoming messages better
  • [STORM-3662] - Continue to support Pacemaker for 2.x topologies
  • [STORM-3667] - Exception message on submitting topology due to incorrect worker max heap size should be more straightforward
  • [STORM-3670] - Separate configurations for daemon metric reporters and topology metrics reporters
  • [STORM-3671] - Add TopologySummary method to NImbus for optimal UI queries
  • [STORM-3673] - update BuiltinMetrics to use v2 Metrics API
  • [STORM-3675] - Eliminate ps command and use /proc/ status in ServerUtils
  • [STORM-3676] - Reduce debug spew to scheduler log
  • [STORM-3682] - Upgrade netty client metrics to use V2 API
  • [STORM-3685] - Detect Loops in Topology at Submit
  • [STORM-3691] - Refactor Resource Aware Strategies (Base, Generic, Default)
  • [STORM-3692] - Handle UID return from Files.getOwner() at process liveness check
  • [STORM-3694] - all V2 metric reporters to report metric short names with dimensions
  • [STORM-3697] - Add metric for capacity
  • [STORM-3706] - Cluster.needsSchedulingRas always succeeds
  • [STORM-3707] - Add meter to track update blob failures
  • [STORM-3708] - ConstraintSolverConfig LOG messages should include topology id
  • [STORM-3709] - Reject topology submission if missing spout
  • [STORM-3714] - Add rate information for TaskMetrics
  • [STORM-3715] - Add Caching to HDFS BlobStore
  • [STORM-3719] - Add configuration for AsyncLocalizer updateBlobs frequency
  • [STORM-3721] - Change child pom.xml reference to parent pom.xml
  • [STORM-3724] - Use blobstore dir modtime to avoid update lookups by HDFSBlobstore
  • [STORM-3737] - Share Worker Metric Registry For Guice AOP Based Metrics Integeration
  • [STORM-3739] - Scheduling should sort numa zones by host groups
  • [STORM-3744] - IntelliJ does not find shaded classes
  • [STORM-3749] - improve logging on server error in StormServerHandler
  • [STORM-3755] - While scheduling multiple ackers with executor use best effort basis
  • [STORM-3758] - Zookeeper reconnect events sometime cause workers to restart
  • [STORM-3774] - Migrate Cgroup metrics to V2
  • [STORM-3778] - convert SpoutThrottlingMetrics to V2 API
  • [STORM-3780] - switch ErrorReportingMetrics to V2 API
  • [STORM-3786] - V2 metrics tick may overreport or not report at all
  • [STORM-3790] - Add meter to track failures WorkerTokenAuthorizer getPassword
  • [STORM-3791] - update metric documentation
  • [STORM-3793] - Add metric to track backpressure status for a task

Bug

  • [STORM-3553] - Upgrade JQuery to 3.5.1
  • [STORM-3624] - Race condition on ArtifactoryConfigLoader.load
  • [STORM-3649] - Logic error regarding storm.supervisor.medium.memory.grace.period.ms
  • [STORM-3652] - Last error not displayed in Topology summary storm ui
  • [STORM-3655] - Worker should commit suicide On Change in its assignment.
  • [STORM-3658] - Problematic worker stays alive because of a deadlock and race condition caused by ShutdownHooks
  • [STORM-3663] - Topology with Mockito 1.x fails to run unit tests
  • [STORM-3666] - Validate component name in rebalance command and fix --executor option
  • [STORM-3677] - Fix Worker Suicide Function if assignment is null
  • [STORM-3679] - misuse of nodeId as hostname in LoadAwareShuffleGrouping
  • [STORM-3684] - receive-queue V2 metrics shouldn't have "_system" as componentId if it is from a system task
  • [STORM-3690] - UI has NullPointerException when the scheduler is not ResourceAwareScheduler
  • [STORM-3695] - Add Timer rates to datapoints for V2 metrics tick
  • [STORM-3696] - ClientSupervisorUtils.processLauncherAndWait ignores InterruptedException
  • [STORM-3699] - fix flight.bash to support flight recording on openJDK8u262 or newer
  • [STORM-3701] - Race Condition between cleanup thread and download tasks
  • [STORM-3704] - Cosmetic: columns shifted in "Topology summary" table
  • [STORM-3712] - Check-Updated-License-Files CI job has been broken
  • [STORM-3720] - BlobStoreFile getModTime() never updates after first call
  • [STORM-3723] - ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result
  • [STORM-3725] - DRPC spout will crash when any one of DRPC server is down
  • [STORM-3727] - SUPERVISOR_SLOTS_PORTS could be list of Longs
  • [STORM-3728] - Workers are not able to connect to Pacemaker if pacemaker.auth.method is KERBEROS
  • [STORM-3729] - Assigning memory greater and equal than 2048m will make assgin memory for slot values to 1m
  • [STORM-3731] - Remove unused nashorn import in storm-loadgen:OutputStream.java
  • [STORM-3733] - AsyncLocalizer stuck looking for missing topology
  • [STORM-3735] - Kyro serialization fails on some metric tuples when topology.fall.back.on.java.serialization is false
  • [STORM-3740] - Asynchronous background blob download can cause orphaned blob references
  • [STORM-3741] - Maven Surefire plugin corrupts output and creates an excessive log file
  • [STORM-3748] - prevent concurrent modification when fetching v2 metrics
  • [STORM-3763] - Backpressure message ignored by the receiver caused the topology to not progress
  • [STORM-3765] - NPE in DRPCSimpleACLAuthorizer.readAclFromConfig when drpc.authorizer.acl has no values
  • [STORM-3767] - NPE on getComponentPendingProfileActions
  • [STORM-3769] - Failed adding references to blobs: FileNotFoundException
  • [STORM-3775] - topology.blobstore.map can cause supervisor restarts
  • [STORM-3785] - Rate metrics are wrongly divided by 1000000

Comment

  • [STORM-3687] - Add a warning about possible issues on a mixed cluster if the StormCommon.systemTopology implementation is changed

Dependency upgrade

Documentation

Test

  • [STORM-3743] - Add new topologies for TestLargeCluster

Sub-task

  • [STORM-3674] - explicitly specify pip2 and pip3 in travis-install.sh
  • [STORM-3681] - Enable basic Travis ARM CI job
  • [STORM-3711] - Enable all the modules in ARM CI