大文件上传后端处理方法
前一篇文章写了大文件上传前端代码,现在记录下后端的处理思路。代码见https://github.com/Mng12345/big-file-upload-backend后台使用springboot实现,主要使用一个数组来判断文件是否完成上传,当每次上传一个信的文件时根据uid在缓存中新建或者将本次的文件块信息记录至缓存中@Getter@Setter@Builderpublic static cla
前一篇文章写了大文件上传前端代码,现在记录下后端的处理思路。代码见
https://github.com/Mng12345/big-file-upload-backend
前端代码见
https://github.com/Mng12345/big-file-upload
后台使用springboot实现,主要使用一个数组来判断文件是否完成上传,当每次上传一个信的文件时根据uid在缓存中新建或者将本次的文件块信息记录至缓存中
@Getter
@Setter
@Builder
public static class FileInfo {
// 路径
private String filePath;
// 文件名
private String fileName;
// uid
private String uid;
private Integer index;
private Integer chunksLength;
private Boolean uploaded;
}
// key为文件的uid,value为Boolean[] 表示每个文件块是否上传完成
private static Map<String, FileInfo[]> fileUploadedCache = new HashMap<>();
当根据数组判断所有文件块都上传完成时,清除缓存中uid对应的数组,并合并文件,清除子文件块。
demo
上传文件块大小限制问题
在测试的时候发现当文件块的大小大于10M时,报org.apache.tomcat.util.http.fileupload.impl.SizeLimitExceededException异常
the request was rejected because its size (10486377) exceeds the configured maximum (10485760)
at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.init(FileItemIteratorImpl.java:150) ~[tomcat-embed-core-9.0.34.jar:9.0.34]
at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.getMultiPartStream(FileItemIteratorImpl.java:194) ~[tomcat-embed-core-9.0.34.jar:9.0.34]
at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.findNextItem(FileItemIteratorImpl.java:213) ~[tomcat-embed-core-9.0.34.jar:9.0.34]
找了下springboot的文档中关于自动配置的类,找到了MultipartAutoConfiguration,文档地址
https://docs.spring.io/spring-boot/docs/2.2.7.RELEASE/reference/html/appendix-auto-configuration-classes.html#auto-configuration-classes
MultipartAutoConfiguration类的具体内容如下:
/*
* Copyright 2012-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.boot.autoconfigure.web.servlet;
import javax.servlet.MultipartConfigElement;
import javax.servlet.Servlet;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication.Type;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.multipart.MultipartResolver;
import org.springframework.web.multipart.commons.CommonsMultipartResolver;
import org.springframework.web.multipart.support.StandardServletMultipartResolver;
import org.springframework.web.servlet.DispatcherServlet;
/**
* {@link EnableAutoConfiguration Auto-configuration} for multi-part uploads. Adds a
* {@link StandardServletMultipartResolver} if none is present, and adds a
* {@link javax.servlet.MultipartConfigElement multipartConfigElement} if none is
* otherwise defined. The {@link ServletWebServerApplicationContext} will associate the
* {@link MultipartConfigElement} bean to any {@link Servlet} beans.
* <p>
* The {@link javax.servlet.MultipartConfigElement} is a Servlet API that's used to
* configure how the server handles file uploads.
*
* @author Greg Turnquist
* @author Josh Long
* @author Toshiaki Maki
* @since 2.0.0
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass({ Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class })
@ConditionalOnProperty(prefix = "spring.servlet.multipart", name = "enabled", matchIfMissing = true)
@ConditionalOnWebApplication(type = Type.SERVLET)
@EnableConfigurationProperties(MultipartProperties.class)
public class MultipartAutoConfiguration {
private final MultipartProperties multipartProperties;
public MultipartAutoConfiguration(MultipartProperties multipartProperties) {
this.multipartProperties = multipartProperties;
}
@Bean
@ConditionalOnMissingBean({ MultipartConfigElement.class, CommonsMultipartResolver.class })
public MultipartConfigElement multipartConfigElement() {
return this.multipartProperties.createMultipartConfig();
}
@Bean(name = DispatcherServlet.MULTIPART_RESOLVER_BEAN_NAME)
@ConditionalOnMissingBean(MultipartResolver.class)
public StandardServletMultipartResolver multipartResolver() {
StandardServletMultipartResolver multipartResolver = new StandardServletMultipartResolver();
multipartResolver.setResolveLazily(this.multipartProperties.isResolveLazily());
return multipartResolver;
}
}
重点关注下面这个配置方法
@Bean
@ConditionalOnMissingBean({ MultipartConfigElement.class, CommonsMultipartResolver.class })
public MultipartConfigElement multipartConfigElement() {
return this.multipartProperties.createMultipartConfig();
}
这里的this.multipartProperties.createMultipartConfig()创建了springboot默认的文件上传的配置参数,跳转到createMultipartConfig方法里可以看到
public MultipartConfigElement createMultipartConfig() {
MultipartConfigFactory factory = new MultipartConfigFactory();
PropertyMapper map = PropertyMapper.get().alwaysApplyingWhenNonNull();
map.from(this.fileSizeThreshold).to(factory::setFileSizeThreshold);
map.from(this.location).whenHasText().to(factory::setLocation);
map.from(this.maxRequestSize).to(factory::setMaxRequestSize);
map.from(this.maxFileSize).to(factory::setMaxFileSize);
return factory.createMultipartConfig();
}
看这里的map.from(this.maxFileSize).to(factory::setMaxFileSize);
继续查看this.maxFileSize看到private DataSize maxFileSize = DataSize.ofMegabytes(1L)
查看ofMegabytes方法,可以看到
public static DataSize ofMegabytes(long megabytes) {
return new DataSize(Math.multiplyExact(megabytes, 1048576L));
}
到这里,终于看到默认配置的上传文件大小限制1048576了,那么怎么解除这个限制呢?有两种办法:
- 在前端分割文件的时候,设置文件块的最大大小要小于1048576
- 在后台代码新建一个配置类,配置类里配置MultipartConfigElement类,代码如下:
@Bean
public MultipartConfigElement multipartConfigElement() {
// 不限制上传文件的大小
return new MultipartConfigElement("");
}
这里MultipartConfigElement("")构造方法内容如下:
public MultipartConfigElement(String location) {
if (location != null) {
this.location = location;
} else {
this.location = "";
}
this.maxFileSize = -1L;
this.maxRequestSize = -1L;
this.fileSizeThreshold = 0;
}
这里设置maxFileSize和maxRequestSize为-1表示不限制上传文件大小,这里的-1表示不限制上传文件大小的原因见org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.init
方法,
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
所有评论(0)