I have encountered a memory management problem in a Spring Boot application, and would like to get advice on how to properly clean up direct memory.
Configuration:
JDK: OpenJDK Runtime Environment (build 21.0.8+9-Ubuntu-0ubuntu122.04.1), OpenJDK 64-Bit Server VM
Spring Boot: Versions 3.2.6 - 3.5.4
I have the following controller and service:
@RequiredArgsConstructor
@RestController
public class MyEntityController {
private final MyEntityService service;
@PostMapping("/file")
public ResponseEntity<String> addedFileInVersion(MultipartHttpServletRequest request) throws IOException {
return service.addFileVersion(request);
}
}
@Service
@Slf4j
@RequiredArgsConstructor
public class MyEntityService {
@Transactional
public ResponseEntity<String> addFileVersion(MultipartHttpServletRequest request) throws IOException {
try {
log.info("Before reading : {} bytes", getDirectMemoryUsed());
MultipartFile file = request.getFile("file");
if (file == null) {
return ResponseEntity.badRequest().body("The file has not been transferred or the file is empty.");
}
byte[] array = file.getBytes();
String fileName = file.getOriginalFilename();
storeFile(array, fileName);
log.info("After reading : {} bytes", getDirectMemoryUsed());
return ResponseEntity.ok("ok");
} catch (IOException e) {
log.error("Error processing file", e);
return ResponseEntity.badRequest().body("Error when uploading file: " + e.GetMessage());
} finally {
log.info("Finally reading : {} bytes", getDirectMemoryUsed());
}
}
private void storeFile(byte[] array, String path) throws IOException {
String decode = UriUtils.decode(path, StandardCharsets.UTF_8.toString());
synchronized (this) {
Path savedPath = Paths.get("/storage/tmp", decode);
if (Files.notExists(savedPath.getParent())) {
Files.createDirectories(savedPath.getParent());
}
Files.write(savedPath, array);
}
}
private double getDirectMemoryUsed() {
List<BufferPoolMXBean> bufferPoolMXBeans = ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class);
for (BufferPoolMXBean pool : bufferPoolMXBeans) {
if ("direct".equals(pool.getName())) {
return pool.getMemoryUsed();
}
}
return 0;
}
}
When sending files, I observe an increase in the use of direct memory, which does not decrease after processing the file. The logs show the following:
Before reading : 24576.0 bytes
After reading : 9.58496462E8 bytes
Finally reading : 9.58496462E8 bytes
Before reading : 9.58512846E8 bytes
After reading : 1.916984732E9 bytes
Finally reading : 1.916984732E9 bytes
It can be seen that after processing the file, the direct memory is not released.
Shouldn't the direct memory be released over time? Because it has been accumulating for several days and has not been cleared and an OutOfMemoryError is being thrown.
Comment From: wilkinsona
It's impossible to say based on the information provided thus far. For example, we don't know what servlet container (embedded or otherwise) you're using. Spring MVC's support for multipart files builds on top of the Servlet API and it's the servlet container that's responsible for allocating memory related to multipart requests.
Regardless of this specific problem, I would not recommend reading the entire uploaded file into memory (as you are doing by calling file.getBytes()
). Instead, I would handle it using getInputStream()
or, probably more simply, transferTo(File)
. If making this change does not help and you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.