Bug report

Environment

  • OpenJDK 22
  • Spring Boot 3.5.4

Process

I have two servers (A and B), both providing a POST /readyz HTTP interface.

I use RestClient + JDK HttpClient + JdkClientHttpRequestFactory to send HTTP requests.

The following simple demo uses a thread to send requests to servers A or B every 10 seconds.

public static RestClient client(String url) {
    HttpClient.Builder httpBuilder = HttpClient.newBuilder()
            .connectTimeout(Duration.ofSeconds(3))
            .version(HttpClient.Version.HTTP_1_1);

    JdkClientHttpRequestFactory factory = new JdkClientHttpRequestFactory(httpBuilder.build());
    factory.setReadTimeout(Duration.ofSeconds(3));
    return RestClient.builder()
            .baseUrl(url)
            .defaultHeader(HttpHeaders.CONNECTION, "closed")
            .defaultHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
            .requestFactory(factory)
            .build();
}

public static void main(String[] args) throws InterruptedException {
    String url = "server.url";
    RestClient client = client(url);

    final ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor(r -> {
        Thread thread = new Thread(r, "node-health-issuer");
        thread.setDaemon(true);
        return thread;
    });

    executor.scheduleWithFixedDelay(() -> {
        try {
            ResponseEntity<Void> responseEntity = client.post().uri("readyz")
                    .retrieve()
                    .toBodilessEntity();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }, 0, 10, TimeUnit.SECONDS);

    Thread.sleep(60 * 1000);
}

Due to version differences in the programs, the responses differ. Server A responds normally, but Server B's response causes the following error:

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://192.1.1.1:11111/readyz": Invalid status line: " 200 OK"
        at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.createResourceAccessException(DefaultRestClient.java:697)
        at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:582)
        at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchange(DefaultRestClient.java:533)
        at org.springframework.web.client.RestClient$RequestHeadersSpec.exchange(RestClient.java:680)
        at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.executeAndExtract(DefaultRestClient.java:814)
        at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toBodilessEntity(DefaultRestClient.java:792)
        at cn.keyou.cmas.entity.monitored.application.health.HealthScheduler.healthLine(HealthScheduler.java:94)
        at cn.keyou.cmas.entity.monitored.application.health.HealthScheduler.lambda$metric$2(HealthScheduler.java:77)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.lang.VirtualThread.run(Unknown Source)
Caused by: java.net.ProtocolException: Invalid status line: " 200 OK"
        at java.net.http/jdk.internal.net.http.Http1HeaderParser.protocolException(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1HeaderParser.readStatusLineFeed(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1HeaderParser.parse(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1Response$Receiver.accept(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.tryAsyncReceive(Unknown Source)
        at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Unknown Source)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(Unknown Source)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(Unknown Source)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

For Server A, using netstat -an | grep <Aport> shows only one established connection, e.g.:

tcp6    0   0 <local-address>   <A-address>     ESTABLISHED

However, for Server B, the error causes connections to accumulate. Before the demo ends, the number of established connections increases continuously, e.g.:

tcp6    0   0 <local-address1>   <B-address>     ESTABLISHED
tcp6    0   0 <local-address2>   <B-address>     ESTABLISHED
tcp6    0   0 <local-address3>   <B-address>     ESTABLISHED
tcp6    0   0 <local-address4>   <B-address>     ESTABLISHED

I noticed in the exception stack trace that in org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:582), there is a finally block. Debugging for Server B shows that close is always true, but clientResponse is null.

I am unsure whether the connections with exceptions should be closed, but they clearly should not keep increasing.

Looking forward to your reply.

Comment From: asifebrahim

Hey, can I take this one?