Dear All, I am facing the below issue when dealing with vertex ai gemini model ... the error seems to be coming from grpc ....

But I am setting this property to REST

spring.ai.vertex.ai.gemini.transport=REST

Knowing that i am using Spring AI version 1.0.0-M3

please help in finding a solution... thanks

com.google.api.gax.rpc.UnknownException: java.net.SocketTimeoutException: Read timed out at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:119) ~[gax-2.49.0.jar:2.49.0] at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41) ~[gax-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonApiExceptionFactory.create(HttpJsonApiExceptionFactory.java:67) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonExceptionResponseObserver.onErrorImpl(HttpJsonExceptionResponseObserver.java:82) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:84) ~[gax-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonDirectStreamController$ResponseObserverAdapter.onClose(HttpJsonDirectStreamController.java:125) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonClientCallImpl$OnCloseNotificationTask.call(HttpJsonClientCallImpl.java:552) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonClientCallImpl.notifyListeners(HttpJsonClientCallImpl.java:391) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonClientCallImpl.deliver(HttpJsonClientCallImpl.java:318) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpJsonClientCallImpl.setResult(HttpJsonClientCallImpl.java:164) ~[gax-httpjson-2.49.0.jar:2.49.0] at com.google.api.gax.httpjson.HttpRequestRunnable.run(HttpRequestRunnable.java:149) ~[gax-httpjson-2.49.0.jar:2.49.0] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na] at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na] at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na] Suppressed: java.lang.RuntimeException: Asynchronous task failed at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105) ~[gax-2.49.0.jar:2.49.0] at com.google.cloud.vertexai.generativeai.ResponseStreamIteratorWithHistory.hasNext(ResponseStreamIteratorWithHistory.java:37) ~[google-cloud-vertexai-1.5.0.jar:1.5.0] at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1855) ~[na:na] at reactor.core.publisher.FluxIterable$IterableSubscription.hasNext(FluxIterable.java:271) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:187) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxStream.subscribe(FluxStream.java:69) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.subscribeInner(FluxSwitchMapNoPrefetch.java:219) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.onNext(FluxSwitchMapNoPrefetch.java:164) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxIterable$IterableSubscription.fastPath(FluxIterable.java:402) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:291) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.onSubscribe(FluxSwitchMapNoPrefetch.java:147) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:201) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxStream.subscribe(FluxStream.java:69) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxPublishOn.subscribeOrReturn(FluxPublishOn.java:92) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:54) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:196) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.publisher.FluxSubscribeOnValue$ScheduledScalar.run(FluxSubscribeOnValue.java:181) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) ~[reactor-core-3.6.10.jar:3.6.10] at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) ~[reactor-core-3.6.10.jar:3.6.10] ... 6 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:288) ~[na:na] at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:314) ~[na:na] at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:355) ~[na:na] at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:808) ~[na:na] at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966) ~[na:na] at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:484) ~[na:na] at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478) ~[na:na] at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) ~[na:na] at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1465) ~[na:na] at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1069) ~[na:na] at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244) ~[na:na] at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:284) ~[na:na] at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343) ~[na:na] at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:826) ~[na:na] at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:761) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1688) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[na:na] at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[na:na] at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[na:na] at com.google.api.client.http.javanet.NetHttpResponse.(NetHttpResponse.java:36) ~[google-http-client-1.44.2.jar:1.44.2] at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:152) ~[google-http-client-1.44.2.jar:1.44.2] at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84) ~[google-http-client-1.44.2.jar:1.44.2] at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.44.2.jar:1.44.2] at com.google.api.gax.httpjson.HttpRequestRunnable.run(HttpRequestRunnable.java:115) ~[gax-httpjson-2.49.0.jar:2.49.0] ... 7 common frames omitted

Comment From: ddobrin

Hi @ account123456789 do you have a reproducer you could share?

Comment From: account123456789

Hi @ account123456789 do you have a reproducer you could share?

Hi @ddobrin ,

I followed the below POC that you have created but with huge payload in JSON format passed in the prompt.

The model takes huge time to respond which is more than the default timeout of spring ai ... which causes the issue.

Any hint about how we can increase the default timeout to huge value ... thanks

https://github.com/ddobrin/gemini-workshop-for-spring-ai-java-developers/blob/main/src/main/java/gemini/workshop/FunctionCallingExample.java

Comment From: emre-safak-wamo

any updates on this? I'm facing the same issue with spring.ai.vertex.ai.gemini.transport=REST set

Comment From: aemresafak

 private VertexAI createVertexAI(String projectId, String projectLocation, String credentialsBase64) {
        var vertexAIBuilder = new VertexAI.Builder()
                .setProjectId(projectId)
                .setTransport(Transport.REST)
                .setLocation(projectLocation);

        vertexAIBuilder.setCredentials(createCredentials(credentialsBase64));
        vertexAIBuilder.setPredictionClientSupplier(this::getPredictionServiceClient);
        return vertexAIBuilder.build();
    }


    private PredictionServiceClient getPredictionServiceClient() {
        var retrySettings = RetrySettings.newBuilder()
                .setTotalTimeoutDuration(Duration.ofMillis(60_000))
                .build();
        var settings = PredictionServiceSettings.newHttpJsonBuilder();

        settings.generateContentSettings()
                .setRetrySettings(retrySettings);

        try {
            return PredictionServiceClient.create(settings.build());
        } catch (IOException e) {
            log.error("Error creating PredictionServiceClient", e);
            throw new RuntimeException(e);
        }
    }

I have found a work-around by supplying the PredictionServiceClient myself. But I think this is a library bug, RestClientCustomizer bean did not help.

Comment From: Elop1

I am also experiencing this issue and am looking for guidance on how to configure the REST timeout for version 1.0.0 GA