Dear All, I am facing the below issue when dealing with vertex ai gemini model ... the error seems to be coming from grpc ....
But I am setting this property to REST
spring.ai.vertex.ai.gemini.transport=REST
Knowing that i am using Spring AI version 1.0.0-M3
please help in finding a solution... thanks
com.google.api.gax.rpc.UnknownException: java.net.SocketTimeoutException: Read timed out
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:119) ~[gax-2.49.0.jar:2.49.0]
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41) ~[gax-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonApiExceptionFactory.create(HttpJsonApiExceptionFactory.java:67) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonExceptionResponseObserver.onErrorImpl(HttpJsonExceptionResponseObserver.java:82) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:84) ~[gax-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonDirectStreamController$ResponseObserverAdapter.onClose(HttpJsonDirectStreamController.java:125) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonClientCallImpl$OnCloseNotificationTask.call(HttpJsonClientCallImpl.java:552) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonClientCallImpl.notifyListeners(HttpJsonClientCallImpl.java:391) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonClientCallImpl.deliver(HttpJsonClientCallImpl.java:318) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpJsonClientCallImpl.setResult(HttpJsonClientCallImpl.java:164) ~[gax-httpjson-2.49.0.jar:2.49.0]
at com.google.api.gax.httpjson.HttpRequestRunnable.run(HttpRequestRunnable.java:149) ~[gax-httpjson-2.49.0.jar:2.49.0]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na]
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Suppressed: java.lang.RuntimeException: Asynchronous task failed
at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105) ~[gax-2.49.0.jar:2.49.0]
at com.google.cloud.vertexai.generativeai.ResponseStreamIteratorWithHistory.hasNext(ResponseStreamIteratorWithHistory.java:37) ~[google-cloud-vertexai-1.5.0.jar:1.5.0]
at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1855) ~[na:na]
at reactor.core.publisher.FluxIterable$IterableSubscription.hasNext(FluxIterable.java:271) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:187) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxStream.subscribe(FluxStream.java:69) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.subscribeInner(FluxSwitchMapNoPrefetch.java:219) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.onNext(FluxSwitchMapNoPrefetch.java:164) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxIterable$IterableSubscription.fastPath(FluxIterable.java:402) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:291) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxSwitchMapNoPrefetch$SwitchMapMain.onSubscribe(FluxSwitchMapNoPrefetch.java:147) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:201) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxStream.subscribe(FluxStream.java:69) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxPublishOn.subscribeOrReturn(FluxPublishOn.java:92) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:54) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:57) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:196) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.publisher.FluxSubscribeOnValue$ScheduledScalar.run(FluxSubscribeOnValue.java:181) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) ~[reactor-core-3.6.10.jar:3.6.10]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) ~[reactor-core-3.6.10.jar:3.6.10]
... 6 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:288) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:314) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:355) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:808) ~[na:na]
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:484) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478) ~[na:na]
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1465) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1069) ~[na:na]
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244) ~[na:na]
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:284) ~[na:na]
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343) ~[na:na]
at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:826) ~[na:na]
at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:761) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1688) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[na:na]
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[na:na]
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[na:na]
at com.google.api.client.http.javanet.NetHttpResponse.
Comment From: ddobrin
Hi @ account123456789 do you have a reproducer you could share?
Comment From: account123456789
Hi @ account123456789 do you have a reproducer you could share?
Hi @ddobrin ,
I followed the below POC that you have created but with huge payload in JSON format passed in the prompt.
The model takes huge time to respond which is more than the default timeout of spring ai ... which causes the issue.
Any hint about how we can increase the default timeout to huge value ... thanks
https://github.com/ddobrin/gemini-workshop-for-spring-ai-java-developers/blob/main/src/main/java/gemini/workshop/FunctionCallingExample.java
Comment From: emre-safak-wamo
any updates on this? I'm facing the same issue with spring.ai.vertex.ai.gemini.transport=REST set
Comment From: aemresafak
private VertexAI createVertexAI(String projectId, String projectLocation, String credentialsBase64) {
var vertexAIBuilder = new VertexAI.Builder()
.setProjectId(projectId)
.setTransport(Transport.REST)
.setLocation(projectLocation);
vertexAIBuilder.setCredentials(createCredentials(credentialsBase64));
vertexAIBuilder.setPredictionClientSupplier(this::getPredictionServiceClient);
return vertexAIBuilder.build();
}
private PredictionServiceClient getPredictionServiceClient() {
var retrySettings = RetrySettings.newBuilder()
.setTotalTimeoutDuration(Duration.ofMillis(60_000))
.build();
var settings = PredictionServiceSettings.newHttpJsonBuilder();
settings.generateContentSettings()
.setRetrySettings(retrySettings);
try {
return PredictionServiceClient.create(settings.build());
} catch (IOException e) {
log.error("Error creating PredictionServiceClient", e);
throw new RuntimeException(e);
}
}
I have found a work-around by supplying the PredictionServiceClient myself. But I think this is a library bug, RestClientCustomizer bean did not help.
Comment From: Elop1
I am also experiencing this issue and am looking for guidance on how to configure the REST timeout for version 1.0.0 GA