前一篇文章写了大文件上传前端代码,现在记录下后端的处理思路。代码见

https://github.com/Mng12345/big-file-upload-backend

前端代码见

https://github.com/Mng12345/big-file-upload

后台使用springboot实现,主要使用一个数组来判断文件是否完成上传,当每次上传一个信的文件时根据uid在缓存中新建或者将本次的文件块信息记录至缓存中

@Getter
@Setter
@Builder
public static class FileInfo {
    // 路径
    private String filePath;
    // 文件名
    private String fileName;
    // uid
    private String uid;
    private Integer index;
    private Integer chunksLength;
    private Boolean uploaded;
}

// key为文件的uid,value为Boolean[] 表示每个文件块是否上传完成
private static Map<String, FileInfo[]> fileUploadedCache = new HashMap<>();

当根据数组判断所有文件块都上传完成时,清除缓存中uid对应的数组,并合并文件,清除子文件块。

demo

在这里插入图片描述

上传文件块大小限制问题

在测试的时候发现当文件块的大小大于10M时,报org.apache.tomcat.util.http.fileupload.impl.SizeLimitExceededException异常

the request was rejected because its size (10486377) exceeds the configured maximum (10485760)
	at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.init(FileItemIteratorImpl.java:150) ~[tomcat-embed-core-9.0.34.jar:9.0.34]
	at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.getMultiPartStream(FileItemIteratorImpl.java:194) ~[tomcat-embed-core-9.0.34.jar:9.0.34]
	at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.findNextItem(FileItemIteratorImpl.java:213) ~[tomcat-embed-core-9.0.34.jar:9.0.34]

找了下springboot的文档中关于自动配置的类,找到了MultipartAutoConfiguration,文档地址

https://docs.spring.io/spring-boot/docs/2.2.7.RELEASE/reference/html/appendix-auto-configuration-classes.html#auto-configuration-classes

MultipartAutoConfiguration类的具体内容如下:

/*
 * Copyright 2012-2019 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.springframework.boot.autoconfigure.web.servlet;

import javax.servlet.MultipartConfigElement;
import javax.servlet.Servlet;

import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication.Type;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.multipart.MultipartResolver;
import org.springframework.web.multipart.commons.CommonsMultipartResolver;
import org.springframework.web.multipart.support.StandardServletMultipartResolver;
import org.springframework.web.servlet.DispatcherServlet;

/**
 * {@link EnableAutoConfiguration Auto-configuration} for multi-part uploads. Adds a
 * {@link StandardServletMultipartResolver} if none is present, and adds a
 * {@link javax.servlet.MultipartConfigElement multipartConfigElement} if none is
 * otherwise defined. The {@link ServletWebServerApplicationContext} will associate the
 * {@link MultipartConfigElement} bean to any {@link Servlet} beans.
 * <p>
 * The {@link javax.servlet.MultipartConfigElement} is a Servlet API that's used to
 * configure how the server handles file uploads.
 *
 * @author Greg Turnquist
 * @author Josh Long
 * @author Toshiaki Maki
 * @since 2.0.0
 */
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass({ Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class })
@ConditionalOnProperty(prefix = "spring.servlet.multipart", name = "enabled", matchIfMissing = true)
@ConditionalOnWebApplication(type = Type.SERVLET)
@EnableConfigurationProperties(MultipartProperties.class)
public class MultipartAutoConfiguration {

	private final MultipartProperties multipartProperties;

	public MultipartAutoConfiguration(MultipartProperties multipartProperties) {
		this.multipartProperties = multipartProperties;
	}

	@Bean
	@ConditionalOnMissingBean({ MultipartConfigElement.class, CommonsMultipartResolver.class })
	public MultipartConfigElement multipartConfigElement() {
		return this.multipartProperties.createMultipartConfig();
	}

	@Bean(name = DispatcherServlet.MULTIPART_RESOLVER_BEAN_NAME)
	@ConditionalOnMissingBean(MultipartResolver.class)
	public StandardServletMultipartResolver multipartResolver() {
		StandardServletMultipartResolver multipartResolver = new StandardServletMultipartResolver();
		multipartResolver.setResolveLazily(this.multipartProperties.isResolveLazily());
		return multipartResolver;
	}

}

重点关注下面这个配置方法

@Bean
@ConditionalOnMissingBean({ MultipartConfigElement.class, CommonsMultipartResolver.class })
public MultipartConfigElement multipartConfigElement() {
	return this.multipartProperties.createMultipartConfig();
}

这里的this.multipartProperties.createMultipartConfig()创建了springboot默认的文件上传的配置参数,跳转到createMultipartConfig方法里可以看到

public MultipartConfigElement createMultipartConfig() {
    MultipartConfigFactory factory = new MultipartConfigFactory();
    PropertyMapper map = PropertyMapper.get().alwaysApplyingWhenNonNull();
    map.from(this.fileSizeThreshold).to(factory::setFileSizeThreshold);
    map.from(this.location).whenHasText().to(factory::setLocation);
    map.from(this.maxRequestSize).to(factory::setMaxRequestSize);
    map.from(this.maxFileSize).to(factory::setMaxFileSize);
    return factory.createMultipartConfig();
}

看这里的map.from(this.maxFileSize).to(factory::setMaxFileSize);继续查看this.maxFileSize看到private DataSize maxFileSize = DataSize.ofMegabytes(1L)查看ofMegabytes方法,可以看到

public static DataSize ofMegabytes(long megabytes) {
    return new DataSize(Math.multiplyExact(megabytes, 1048576L));
}

到这里,终于看到默认配置的上传文件大小限制1048576了,那么怎么解除这个限制呢?有两种办法:

  1. 在前端分割文件的时候,设置文件块的最大大小要小于1048576
  2. 在后台代码新建一个配置类,配置类里配置MultipartConfigElement类,代码如下:
@Bean
public MultipartConfigElement multipartConfigElement() {
    // 不限制上传文件的大小
    return new MultipartConfigElement("");
}

这里MultipartConfigElement("")构造方法内容如下:

public MultipartConfigElement(String location) {
    if (location != null) {
        this.location = location;
    } else {
        this.location = "";
    }

    this.maxFileSize = -1L;
    this.maxRequestSize = -1L;
    this.fileSizeThreshold = 0;
}

这里设置maxFileSize和maxRequestSize为-1表示不限制上传文件大小,这里的-1表示不限制上传文件大小的原因见org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.init方法,
在这里插入图片描述

Logo

开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!

更多推荐