If you are a C# developer and apply asp.net core to your project, do you know how ASP.NET Core reads Request.Body? When we use ASP.NET Core to develop projects, we will definitely involve the scene of reading Request.Body, because most POST requests store data in the Http Body. The main thing I use in my daily development is ASP.NET Core, so I often encounter this kind of scene. This post is to share the reading problem of Request.Body.
ASP.NET Core reads Request.Body
When we want to read the Request Body, the following code is a common way of writing. Here we simulate reading the Request Body in the Filter, reading similarly in Action or Middleware or other places, where there is a Request, there is a Body, as shown below Show:
public override void OnActionExecuting(ActionExecutingContext context)
{
//Request Body in ASP.NET Core is in the form of Stream
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = stream.ReadToEnd();
_logger.LogDebug("body content:" + body);
base.OnActionExecuting(context);
}
How ASP.NET Core reads Request.Body
Unfortunately, after you write this code, you will find this error System.InvalidOperationException: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead.
Synchronous reading
First of all, let’s look at the way to set AllowSynchronousIO to true. Look at the name and know that it is to allow synchronous IO. There are roughly two ways to set it. Later we will explore the direct differences between them through the source code. Let’s take a look at how to set AllowSynchronousIO. Value. The first way is to configure in ConfigureServices, the operation is as follows
services.Configure<KestrelServerOptions>(options =>
{
options.AllowSynchronousIO = true;
});
How ASP.NET Core reads Request.Body
This method is the same as configuring the Kestrel option configuration in the configuration file, but the method is different. After the setting is completed, it will not report an error when running. There is another way, which can be set by IHttpBodyControlFeature without setting it in ConfigureServices, as follows:
public override void OnActionExecuting(ActionExecutingContext context)
{
var syncIOFeature = context.HttpContext.Features.Get<IHttpBodyControlFeature>();
if (syncIOFeature != null)
{
syncIOFeature.AllowSynchronousIO = true;
}
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = stream.ReadToEnd();
_logger.LogDebug("body content:" + body);
base.OnActionExecuting(context);
}
How ASP.NET Core reads Request.Body
This method is also effective. In this way, you don’t need to set it every time you read the Body, just set it once before you are ready to read the Body. Both of these methods are to set AllowSynchronousIO to true, but we need to think about why Microsoft set AllowSynchronousIO to false by default, indicating that Microsoft does not want us to read the Body synchronously. I came to such a conclusion by searching for information
Kestrel: AllowSynchronousIO (synchronous IO) is disabled by default. Insufficient threads can cause application crashes. Synchronous I/O APIs (such as HttpRequest.Body.Read) are a common cause of insufficient threads.
It can be known that although this method can solve the problem, the performance is not bad. Microsoft does not recommend this operation. When the program traffic is relatively large, it is easy to cause the program to become unstable or even crash.
Asynchronous read
From the above we learned that Microsoft does not want us to operate by setting AllowSynchronousIO, because it will affect performance. Then we can use the asynchronous method to read, the asynchronous method mentioned here is actually to use the asynchronous method that comes with the Stream to read, as shown below
public override void OnActionExecuting(ActionExecutingContext context)
{
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = stream.ReadToEndAsync().GetAwaiter().GetResult();
_logger.LogDebug("body content:" + body);
base.OnActionExecuting(context);
}
How ASP.NET Core reads Request.Body
It’s that simple, there is no need to set up other things, just through the asynchronous method of ReadToEndAsync to operate. Many operations in ASP.NET Core are asynchronous operations, even filters or middleware can directly return Task type methods, so we can directly use asynchronous operations
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = await stream.ReadToEndAsync();
_logger.LogDebug("body content:" + body);
await next();
}
How ASP.NET Core reads Request.Body
The advantage of these two methods of operation is that there is no need to set anything else, just read through the asynchronous method, which is also our recommended practice. The more magical thing is that we just replaced the ReadToEnd of the StreamReader with the ReadToEndAsync method and everyone is happy, do you feel more magical? When we feel magical, it is because we don’t know enough about it. Next, we will uncover its mystery step by step through the source code.
Repeat reading
Above we demonstrated the use of synchronous and asynchronous methods to read RequestBody, but is this really enough? In fact, it does not work. In this way, the correct Body result can only be read once per request. If you continue to read the RequestBody Stream, you will not be able to read any content. Let’s take an example.
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = await stream.ReadToEndAsync();
_logger.LogDebug("body content:" + body);
StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
string body2 = await stream2.ReadToEndAsync();
_logger.LogDebug("body2 content:" + body2);
await next();
}
How ASP.NET Core reads Request.Body
In the above example, there is a correct RequestBody result in body, but an empty string in body2. This situation is relatively bad, why do you say that? If you read the RequestBody in Middleware, and the execution of this middleware is before the model binding, then the model binding will fail, because sometimes the model binding also needs to read the RequestBody to get the http request content.
As for why we believe this, everyone has a certain understanding, because after we read the Stream, the position of the Stream pointer at this time is already at the end of the Stream, that is, Position is not 0 at this time, and Stream reading depends on Position. To mark where the external stream is read, so when we read it again, we will start reading from the end, and we won’t be able to read any information. So if we want to read the RequestBody repeatedly, we must reset the Position of the RequestBody to 0 before reading it again, as shown below
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = await stream.ReadToEndAsync();
_logger.LogDebug("body content:" + body);
//Or use reset Position context.HttpContext.Request.Body.Position = 0;
//If you are sure that the Position has been reset after the last reading, then this sentence can be omitted
context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
string body2 = await stream2.ReadToEndAsync();
//We'll try to reset it as much as possible
context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
_logger.LogDebug("body2 content:" + body2);
await next();
}
How ASP.NET Core reads Request.Body
After writing, I happily run it to see the effect, and found that an error was reported
System.NotSupportedException: Specified method is not supported.at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Seek(Int64 offset , SeekOrigin origin)
How ASP.NET Core reads Request.Body
It is generally understandable that this operation is not supported. As for why, let’s take a look when we parse the source code. Having said that, how can we solve it? It’s also very simple. Microsoft knows its own bugs and naturally provides us with a solution. It is also very simple to use by adding EnableBuffering
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
//Add EnableBuffering before operating Request.Body
context.HttpContext.Request.EnableBuffering();
StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
string body = await stream.ReadToEndAsync();
_logger.LogDebug("body content:" + body);
context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
//Attention here! ! ! I have used synchronous reading
string body2 = stream2.ReadToEnd();
context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
_logger.LogDebug("body2 content:" + body2);
await next();
}
How ASP.NET Core reads Request.Body
By adding Request.EnableBuffering() we can read the RequestBody repeatedly. Looking at the name, we can guess it roughly. It is related to the cached RequestBody. It should be noted Request.EnableBuffering() that it will have an effect before the RequestBody is ready to be read, otherwise it will be invalid You only need to add it once per request. And everyone saw that when I read the Body for the second time, I used the synchronous method to read the RequestBody. Is it amazing? We will analyze this problem from the perspective of the source code later in the meeting.
Source code research
Above we see through StreamReadera ReadToEnd synchronous read Request.Body needs to be set AllowSynchronousIO as true to operate, but the use of StreamReaderthe ReadToEndAsync method it can be directly manipulated.
The relationship between StreamReader and Stream
We have seen that it’s all through the operation StreamReadermethod. It’s about my Request.Body. Don’t worry. Let’s take a look at the operation here. First, let’s take a rough look at ReadToEndthe implementation to understand StreamReaderwhat is related to the Stream. Find ReadToEnd Method [ click to view source code👈 ]
public override string ReadToEnd()
{
ThrowIfDisposed();
CheckAsyncTaskInProgress();
// Call ReadBuffer, and then extract data from charBuffer.
StringBuilder sb = new StringBuilder(_charLen-_charPos);
do
{
//Circular stitching to read the content
sb.Append(_charBuffer, _charPos, _charLen-_charPos);
_charPos = _charLen;
//Read the buffer, this is the core operation
ReadBuffer();
} while (_charLen> 0);
//Return to read content
return sb.ToString();
}
How ASP.NET Core reads Request.Body
Through this source code, we have learned such information. One is StreamReader that the ReadToEnd essence is to read the ReadBuffer through a loop and then stitch the read content through StringBuilder. The core is to read the ReadBuffer method. Since there are more codes, we find the core Operation [ click to view source code👈 ]
if (_checkPreamble)
{
//From here we can know that the essence is to use the Read method in the Stream to be read
int len = _stream.Read(_byteBuffer, _bytePos, _byteBuffer.Length-_bytePos);
if (len == 0)
{
if (_byteLen> 0)
{
_charLen += _decoder.GetChars(_byteBuffer, 0, _byteLen, _charBuffer, _charLen);
_bytePos = _byteLen = 0;
}
return _charLen;
}
_byteLen += len;
}
else
{
//From here we can know that the essence is to use the Read method in the Stream to be read
_byteLen = _stream.Read(_byteBuffer, 0, _byteBuffer.Length);
if (_byteLen == 0)
{
return _charLen;
}
}
How ASP.NET Core reads Request.Body
Through the above code, we can understand that StreamReader it is actually a tool class, which just encapsulates the original operation on the Stream, and the ReadToEndessence of simplifying our code method is the reading Stream的Readmethod. Next, let’s take a look at ReadToEndAsync the specific implementation of the method [ click to view the source code👈 ]
public override Task<string> ReadToEndAsync()
{
if (GetType() != typeof(StreamReader))
{
return base.ReadToEndAsync();
}
ThrowIfDisposed();
CheckAsyncTaskInProgress();
//The essence is the ReadToEndAsyncInternal method
Task<string> task = ReadToEndAsyncInternal();
_asyncReadTask = task;
return task;
}
private async Task<string> ReadToEndAsyncInternal()
{
//It is also the content read by loop stitching
StringBuilder sb = new StringBuilder(_charLen-_charPos);
do
{
int tmpCharPos = _charPos;
sb.Append(_charBuffer, tmpCharPos, _charLen-tmpCharPos);
_charPos = _charLen;
//The core operation is the ReadBufferAsync method
await ReadBufferAsync(CancellationToken.None).ConfigureAwait(false);
} while (_charLen> 0);
return sb.ToString();
}
How ASP.NET Core reads Request.Body
Through this we can see that the core operation is the ReadBufferAsync method, and there are more codes. Let’s also look at the core implementation [ click to view the source code👈 ]
byte[] tmpByteBuffer = _byteBuffer;
//Stream assignment to tmpStream
Stream tmpStream = _stream;
if (_checkPreamble)
{
int tmpBytePos = _bytePos;
//The essence is to call the ReadAsync method of Stream
int len = await tmpStream.ReadAsync(new Memory<byte>(tmpByteBuffer, tmpBytePos, tmpByteBuffer.Length-tmpBytePos), cancellationToken).ConfigureAwait(false);
if (len == 0)
{
if (_byteLen> 0)
{
_charLen += _decoder.GetChars(tmpByteBuffer, 0, _byteLen, _charBuffer, _charLen);
_bytePos = 0; _byteLen = 0;
}
return _charLen;
}
_byteLen += len;
}
else
{
//The essence is to call the ReadAsync method of Stream
_byteLen = await tmpStream.ReadAsync(new Memory<byte>(tmpByteBuffer), cancellationToken).ConfigureAwait(false);
if (_byteLen == 0)
{
return _charLen;
}
}
How ASP.NET Core reads Request.Body
Through the above code, I can understand that the essence of StreamReader is to read the packaging of the Stream, and the core method still comes from the Stream itself. The reason why we introduced the StreamReader class in general is to show you the relationship between StreamReader and Stream. Otherwise, we are afraid that everyone will misunderstand that this wave of operations is the implementation of StreamReader, not the problem of Request.Body. In fact, it is not all like this. Everything is pointing to Stream, The Body of Request is Stream you can check it out for yourself, and we can continue when we understand this step.
HttpRequest’s Body
We mentioned above that the essence of Request’s Body is Stream, and Stream itself is an abstract class, so Request.Body is an implementation class of Stream. By default, Request.Body is an instance of HttpRequestStream [ click to view the source code 👈 ], what we said here is the default, because it can be changed, we will talk about it later. We get from the above conclusion of StreamReader that the essence of ReadToEnd is the Read method of the called Stream, that is, the Read method of HttpRequestStream here. Let’s take a look at the specific implementation [ click to view the source code👈 ]
public override int Read(byte[] buffer, int offset, int count)
{
//Do you know why it reported an error when reading the Body synchronously?
if (!_bodyControl.AllowSynchronousIO)
{
throw new InvalidOperationException(CoreStrings.SynchronousReadsDisallowed);
}
//The essence is to call ReadAsync
return ReadAsync(buffer, offset, count).GetAwaiter().GetResult();
}
How ASP.NET Core reads Request.Body
Through this code, we can know why reading the Body without setting AllowSynchronousIO to true will throw an exception. This is a program-level control, and we also understand that the essence of Read is to call ReadAsync asynchronously. method
public override ValueTask<int> ReadAsync(Memory<byte> destination, CancellationToken cancellationToken = default)
{
return ReadAsyncWrapper(destination, cancellationToken);
}
How ASP.NET Core reads Request.Body
ReadAsync itself has no special restrictions, so there will be no exceptions similar to Read when directly operating ReadAsync.
Through this we came to the conclusion that Request.Body, that is, the synchronous read of HttpRequestStream will throw an exception, while the asynchronous read of ReadAsync will not throw an exception. It is only related to the value of AllowSynchronousIO in the Read method of HttpRequestStream itself.
The essential source of AllowSynchronousIO
Through the Read method of HttpRequestStream, we can know that AllowSynchronousIO controls the way of synchronous reading. And we also learned that AllowSynchronousIO has several different ways to configure. Next, let’s take a look at the essence of the several ways. Through HttpRequestStream we know that the attribute of AllowSynchronousIO in the Read method comes from IHttpBodyControlFeature, which is the second configuration method we introduced above
private readonly HttpRequestPipeReader _pipeReader;
private readonly IHttpBodyControlFeature _bodyControl;
public HttpRequestStream(IHttpBodyControlFeature bodyControl, HttpRequestPipeReader pipeReader)
{
_bodyControl = bodyControl;
_pipeReader = pipeReader;
}
How ASP.NET Core reads Request.Body
Then it KestrelServerOptions has something to do with it, because we only configure KestrelServerOptions for HttpRequestStream’s Read to not report exceptions, and HttpRequestStream’s Read only relies on the AllowSynchronousIO property of IHttpBodyControlFeature. The initialization of HttpRequestStream in Kestrel is in BodyControl [ click to view source code👈 ]
private readonly HttpRequestStream _request;
public BodyControl(IHttpBodyControlFeature bodyControl, IHttpResponseControl responseControl)
{
_request = new HttpRequestStream(bodyControl, _requestReader);
}
How ASP.NET Core reads Request.Body
The place to initialize BodyControl is in HttpProtocol, we find the InitializeBodyControl method to initialize BodyControl [ click to view source code👈 ]
public void InitializeBodyControl(MessageBody messageBody)
{
if (_bodyControl == null)
{
//What is passed here is bodyControl and what is passed is this
_bodyControl = new BodyControl(bodyControl: this, this);
}
(RequestBody, ResponseBody, RequestBodyPipeReader, ResponseBodyPipeWriter) = _bodyControl.Start(messageBody);
_requestStreamInternal = RequestBody;
_responseStreamInternal = ResponseBody;
}
How ASP.NET Core reads Request.Body
Here we can see that the initial IHttpBodyControlFeature is passed this, which is the current instance of HttpProtocol. In other words, HttpProtocol implements the IHttpBodyControlFeature interface, and HttpProtocol itself is partial. We can see the implementation relationship in one of the distribution classes HttpProtocol.FeatureCollection
[ click to view source code👈 ]
internal partial class HttpProtocol : IHttpRequestFeature,
IHttpRequestBodyDetectionFeature,
IHttpResponseFeature,
IHttpResponseBodyFeature,
IRequestBodyPipeFeature,
IHttpUpgradeFeature,
IHttpConnectionFeature,
IHttpRequestLifetimeFeature,
IHttpRequestIdentifierFeature,
IHttpRequestTrailersFeature,
IHttpBodyControlFeature,
IHttpMaxRequestBodySizeFeature,
IEndpointFeature,
IRouteValuesFeature
{
bool IHttpBodyControlFeature.AllowSynchronousIO
{
get => AllowSynchronousIO;
set => AllowSynchronousIO = value;
}
}
How ASP.NET Core reads Request.Body
Through this, we can see that HttpProtocol does implement the IHttpBodyControlFeature interface. Next, we find the place to initialize AllowSynchronousIO, find AllowSynchronousIO = ServerOptions.AllowSynchronousIO;this code description comes from the attribute of ServerOptions, and find the place to initialize ServerOptions [ click to view source code👈 ]
private HttpConnectionContext _context;
//ServiceContext initialization comes from HttpConnectionContext
public ServiceContext ServiceContext => _context.ServiceContext;
protected KestrelServerOptions ServerOptions {get; set;} = default!;
public void Initialize(HttpConnectionContext context)
{
_context = context;
//From ServiceContext
ServerOptions = ServiceContext.ServerOptions;
Reset();
HttpResponseControl = this;
}
How ASP.NET Core reads Request.Body
Through this we know that ServerOptions comes from the ServerOptions attribute of ServiceContext, we find the place to assign value to ServiceContext, in the CreateServiceContext method of KestrelServerImpl [ click to view the source code👈 ] to simplify the logic, extract the core content and roughly implement the following
public KestrelServerImpl(
IOptions<KestrelServerOptions> options,
IEnumerable<IConnectionListenerFactory> transportFactories,
ILoggerFactory loggerFactory)
//The injected IOptions<KestrelServerOptions> called CreateServiceContext
: this(transportFactories, null, CreateServiceContext(options, loggerFactory))
{
}
private static ServiceContext CreateServiceContext(IOptions<KestrelServerOptions> options, ILoggerFactory loggerFactory)
{
//The value comes from IOptions<KestrelServerOptions>
var serverOptions = options.Value ?? new KestrelServerOptions();
return new ServiceContext
{
Log = trace,
HttpParser = new HttpParser<Http1ParsingHandler>(trace.IsEnabled(LogLevel.Information)),
Scheduler = PipeScheduler.ThreadPool,
SystemClock = heartbeatManager,
DateHeaderValueManager = dateHeaderValueManager,
ConnectionManager = connectionManager,
Heartbeat = heartbeat,
//Assignment operation
ServerOptions = serverOptions,
};
}
How ASP.NET Core reads Request.Body
Through the above code, we can see that if KestrelServerOptions is configured, then the ServerOptions property of ServiceContext comes from KestrelServerOptions, that is, we have services.Configure<KestrelServerOptions>()obtained such a conclusion through the configuration value.
If KestrelServerOptions is configured, services.Configure(), then AllowSynchronousIO comes from KestrelServerOptions. That is, the AllowSynchronousIO attribute of IHttpBodyControlFeature comes from KestrelServerOptions. If there is no configuration, the
same effect can be obtained by directly modifying the AllowSynchronousIO property of the IHttpBodyControlFeature instance. After all, HttpRequestStream is the directly dependent IHttpBodyControlFeature instance.
Behind the magic of EnableBuffering
We have seen in the above example that if you don’t add EnableBuffering, you can directly set the Position of RequestBody to report an error of NotSupportedException, and after adding it, I can directly use the synchronous method to read RequestBody. First of all, let’s take a look at What will report an error, we know from the above error that the error comes from the HttpRequestStream class [ click to view the source code 👈 ], above we also said that this class inherits the Stream abstract class, through the source code we can see the following related code
//Cannot use Seek operation
public override bool CanSeek => false;
//Allow reading
public override bool CanRead => true;
//Not allowed to write
public override bool CanWrite => false;
//Cannot get the length
public override long Length => throw new NotSupportedException();
//Cannot read and write Position
public override long Position
{
get => throw new NotSupportedException();
set => throw new NotSupportedException();
}
//Cannot use Seek method
public override long Seek(long offset, SeekOrigin origin)
{
throw new NotSupportedException();
}
How ASP.NET Core reads Request.Body
I believe that through these we can clearly see that the setting or writing-related operations for HttpRequestStream are not allowed. This is why we will report an error when we set the Position directly through Seek above, and there are some other operating restrictions. In short, the default is that we do not want us to do too many operations on HttpRequestStream, especially setting or writing related operations. But when we use EnableBuffering, we don’t have these problems. Why?
Next we have to uncover what it is. First of all, we start with Request.EnableBuffering() this method, find the source code location in the HttpRequestRewindExtensions extension class [ click to view the source code👈 ], we start with the simplest parameterless method and see the following definition
/// <summary>
/// Ensure that Request.Body can be read multiple times
/// </summary>
/// <param name="request"></param>
public static void EnableBuffering(this HttpRequest request)
{
BufferingHelper.EnableRewind(request);
}
How ASP.NET Core reads Request.Body
The above method is the simplest form, and there is an extension method of EnableBuffering that is the extension method with the most complete parameters. This method can control the size of the read and the limited size of whether to store to the disk.
/// <summary>
/// Ensure that Request.Body can be read multiple times
/// </summary>
/// <param name="request"></param>
/// <param name="bufferThreshold"> The maximum size (bytes) used to buffer the stream in the memory. The larger request body is written to disk. </param>
/// <param name="bufferLimit">The maximum size (bytes) of the request body. Attempting to read beyond this limit will result in an exception</param>
public static void EnableBuffering(this HttpRequest request, int bufferThreshold, long bufferLimit)
{
BufferingHelper.EnableRewind(request, bufferThreshold, bufferLimit);
}
How ASP.NET Core reads Request.Body
Regardless of the form, BufferingHelper.EnableRewind this method is ultimately called . Not much to say, directly find the BufferingHelper class and find the location of the class [ Click to view the source code 👈 ] The code is not much and relatively simple, let’s paste the implementation of EnableRewind
//The cacheable size in the default memory is 30K, and if it exceeds this size, it will be stored to disk
internal const int DefaultBufferThreshold = 1024 * 30;
/// <summary>
/// This method is also an extension method of HttpRequest
/// </summary>
/// <returns></returns>
public static HttpRequest EnableRewind(this HttpRequest request, int bufferThreshold = DefaultBufferThreshold, long? bufferLimit = null)
{
if (request == null)
{
throw new ArgumentNullException(nameof(request));
}
//Get the Request Body first
var body = request.Body;
//By default, the Body is HttpRequestStream and CanSeek is false, so it will definitely be executed in the if logic
if (!body.CanSeek)
{
//Instantiated the FileBufferingReadStream class, it seems that this is the key
var fileStream = new FileBufferingReadStream(body, bufferThreshold,bufferLimit,AspNetCoreTempDirectory.TempDirectoryFactory);
//Assign to Body, which means that the Request.Body type will be FileBufferingReadStream after EnableBuffering is turned on
request.Body = fileStream;
//Here to register fileStream to Response for easy release
request.HttpContext.Response.RegisterForDispose(fileStream);
}
return request;
}
How ASP.NET Core reads Request.Body
From the above source code implementation, we can roughly draw two conclusions
- The EnableRewind method of BufferingHelper is also an extension method of HttpRequest, which can be
Request.EnableRewindcalled directly through the form, the effect is equivalent to the callRequest.EnableBufferingbecause EnableBuffering is also called EnableRewind - After enabling the EnableBuffering operation, FileBufferingReadStream will actually be used to replace the default HttpRequestStream, so the subsequent processing of RequestBody will be an instance of FileBufferingReadStream
Through the above analysis, we can clearly see that the core operation lies in FileBufferingReadStream this class, and it can be seen from the name that it must also inherit the Stream abstract class, so what are you waiting for to find the implementation of FileBufferingReadStream [ click to view the source code👈 ], First look at the definition of other classes
public class FileBufferingReadStream : Stream
{
}
How ASP.NET Core reads Request.Body
Undoubtedly it is indeed inherited from the Steam class. We have also seen above that the RequestBody can be set and read repeatedly after using Request.EnableBuffering, indicating that some rewriting operations have been carried out. Let’s take a look.
/// <summary>
/// Allow reading
/// </summary>
public override bool CanRead
{
get {return true;}
}
/// <summary>
/// Allow Seek
/// </summary>
public override bool CanSeek
{
get {return true;}
}
/// <summary>
/// Write is not allowed
/// </summary>
public override bool CanWrite
{
get {return false;}
}
/// <summary>
/// You can get the length
/// </summary>
public override long Length
{
get {return _buffer.Length;}
}
/// <summary>
/// Can read and write Position
/// </summary>
public override long Position
{
get {return _buffer.Position;}
set
{
ThrowIfDisposed();
_buffer.Position = value;
}
}
public override long Seek(long offset, SeekOrigin origin)
{
//If the Body has been released, it will be abnormal
ThrowIfDisposed();
//Throw an exception under special circumstances
//_completelyBuffered represents whether it is fully buffered or not must be set to true after the original HttpRequestStream is read
//If the reading is not completed but the original position information is inconsistent with the current position information, an exception will be thrown directly
if (!_completelyBuffered && origin == SeekOrigin.End)
{
throw new NotSupportedException("The content has not been fully buffered yet.");
}
else if (!_completelyBuffered && origin == SeekOrigin.Current && offset + Position> Length)
{
throw new NotSupportedException("The content has not been fully buffered yet.");
}
else if (!_completelyBuffered && origin == SeekOrigin.Begin && offset> Length)
{
throw new NotSupportedException("The content has not been fully buffered yet.");
}
//Seek to recharge buffer
return _buffer.Seek(offset, origin);
}
How ASP.NET Core reads Request.Body
Because some key settings have been rewritten, we can set some stream-related operations. From the Seek method we see two important parameters _completelyBufferedand _buffer, _completelyBuffered used to determine whether the original HttpRequestStream reading is completed, the final analysis, or because FileBufferingReadStream first read the contents of HttpRequestStream. _buffer is the content read from HttpRequestStream. Let’s take a look at the logic and remember that this is not all the logic, but the general idea.
private readonly ArrayPool<byte> _bytePool;
private const int _maxRentedBufferSize = 1024 * 1024; //1MB
private Stream _buffer;
public FileBufferingReadStream(int memoryThreshold)
{
//Even if we set the memoryThreshold, it cannot exceed 1MB or it will be stored on the disk
if (memoryThreshold <= _maxRentedBufferSize)
{
_rentedBuffer = bytePool.Rent(memoryThreshold);
_buffer = new MemoryStream(_rentedBuffer);
_buffer.SetLength(0);
}
else
{
//More than 1M will be cached to disk, so only initialization
_buffer = new MemoryStream();
}
}
How ASP.NET Core reads Request.Body
These are some initial operations. Of course, the core operation is still in the Read method of FileBufferingReadStream, because the real reading place is here, we find the position of the Read method [ click to view the source code👈 ]
private readonly Stream _inner;
public FileBufferingReadStream(Stream inner)
{
//Receive the original Request.Body
_inner = inner;
}
public override int Read(Span<byte> buffer)
{
ThrowIfDisposed();
//If the reading is completed, get the information directly in the buffer and return directly
if (_buffer.Position <_buffer.Length || _completelyBuffered)
{
return _buffer.Read(buffer);
}
//I will not come here until the reading is completed
//_inner is the original RequestBody received
//The RequestBody read is put into the buffer
var read = _inner.Read(buffer);
//If the set length is exceeded, an exception will be thrown
if (_bufferLimit.HasValue && _bufferLimit-read <_buffer.Length)
{
throw new IOException("Buffer limit exceeded.");
}
//If the setting is stored in the memory and the body length is greater than the set length that can be stored in the memory, it is stored in the disk
if (_inMemory && _memoryThreshold-read <_buffer.Length)
{
_inMemory = false;
//Cache the original Body stream
var oldBuffer = _buffer;
//Create cache file
_buffer = CreateTempFile();
//Exceeded the memory storage limit, but has not written a temporary file
if (_rentedBuffer == null)
{
oldBuffer.Position = 0;
var rentedBuffer = _bytePool.Rent(Math.Min((int)oldBuffer.Length, _maxRentedBufferSize));
try
{
//Read the Body stream into the cache file stream
var copyRead = oldBuffer.Read(rentedBuffer);
//Judging whether to read to the end
while (copyRead> 0)
{
//Write oldBuffer to the buffer file stream _buffer
_buffer.Write(rentedBuffer.AsSpan(0, copyRead));
copyRead = oldBuffer.Read(rentedBuffer);
}
}
finally
{
//Return the temporary buffer to ArrayPool after reading
_bytePool.Return(rentedBuffer);
}
}
else
{
_buffer.Write(_rentedBuffer.AsSpan(0, (int)oldBuffer.Length));
_bytePool.Return(_rentedBuffer);
_rentedBuffer = null;
}
}
//If the RequestBody is not read to the end, it will be written to the buffer
if (read> 0)
{
_buffer.Write(buffer.Slice(0, read));
}
else
{
//Update _completelyBuffered if the RequestBody has been read, that is, it is written to the buffer.
//Mark as complete reading RequestBody, and then read RequestBody directly in _buffer
_completelyBuffered = true;
}
//Returns the number of bytes read to be used by the external StreamReader to determine whether the reading is complete
return read;
}
How ASP.NET Core reads Request.Body
The code is more and more complicated. In fact, the core idea is still relatively clear. Let’s summarize it roughly.
- First determine whether the original RequestBody has been completely read, and if the RequestBody has been completely read, it will be returned directly in the buffer.
- If the RequestBody length is greater than the set memory storage limit, the buffer will be written to the disk temporary file
- If it is the first time to read or complete the RequestBody for a complete read, then write the content of the RequestBody to the buffer and know that the read is complete
Among them, CreateTempFile is the operation flow of creating a temporary file. The purpose is to write the RequestBody information into the temporary file. You can specify the address of the temporary file. If you don’t specify it, the system default directory will be used. Its implementation is as follows [ click to view the source code👈 ]
private Stream CreateTempFile()
{
//Determine whether the cache directory has been established, if not, use the system temporary file directory
if (_tempFileDirectory == null)
{
Debug.Assert(_tempFileDirectoryAccessor != null);
_tempFileDirectory = _tempFileDirectoryAccessor();
Debug.Assert(_tempFileDirectory != null);
}
//The full path of the temporary file
_tempFileName = Path.Combine(_tempFileDirectory, "ASPNETCORE_" + Guid.NewGuid().ToString() + ".tmp");
//Return the operation stream of the temporary file
return new FileStream(_tempFileName, FileMode.Create, FileAccess.ReadWrite, FileShare.Delete, 1024 * 16,
FileOptions.Asynchronous | FileOptions.DeleteOnClose | FileOptions.SequentialScan);
}
How ASP.NET Core reads Request.Body
We have analyzed the Read method of FileBufferingReadStream above. This method is a synchronous reading method for the ReadToEnd method of StreamReader. Of course, it also has an asynchronous reading method ReadAsync for the ReadToEndAsync method of StreamReader. The implementation logic of these two methods is completely the same, but the read and write operations are asynchronous operations, we will not introduce that method here, and interested students can learn about ReadAsync the implementation of the method by themselves [ click to view the source code👈 ]
When EnableBuffering is turned on, regardless of whether the first read is the ReadToEnd synchronous read method with AllowSynchronousIO set to true, or the asynchronous read method of ReadToEndAsync is used directly, then the ReadToEnd synchronous method is used again to read the Request.Body and there is no need to set it. AllowSynchronousIO is true. Because the default Request.Body has been replaced by an HttpRequestStream instance with a FileBufferingReadStream instance, and FileBufferingReadStream rewrites the Read and ReadAsync methods, there is no restriction that does not allow synchronous reading.
Conclusion
This article has a lot of space. If you want to study the relevant logic in depth, I hope this article can give you some guidance on reading the source code. In order to prevent you from delving into the article and forgetting the specific process logic, here we will summarize all the conclusions about the correct reading of RequestBody
- First of all, about synchronous reading.
Request.BodyBecause the default RequestBody implementation is HttpRequestStream, HttpRequestStream will determine whether to enable AllowSynchronousIO when rewriting the Read method, and throw an exception directly if it is not enabled. But the ReadAsync method of HttpRequestStream does not have this restriction, so there is no exception to the asynchronous method of reading RequestBody. - Although we can read the RequestBody by setting
AllowSynchronousIOor usingReadAsyncit, the RequestBody cannot be read repeatedly. This is because the Position and Seek of HttpRequestStream are not allowed to modify operations, and an exception will be thrown directly if they are set. In order to be able to read repeatedly, we have introduced the Request extension method.EnableBufferingThrough this method, we can reset the reading position to achieve repeated reading of RequestBody. - The opening
EnableBufferingmethod can be set once per request, that is, set before the RequestBody is ready to be read. Its essence is actually to useFileBufferingReadStreamthe default type HttpRequestStream instead of the default RequestBody, so that when we manipulate the Body in an Http request, we actually operate FileBufferingReadStream. When this class rewrites Stream, both Position and Seek can be set, so that we can achieve it. Repeat reading. FileBufferingReadStreamNot only does it bring us repeatable reading, but it also adds a caching function to the RequestBody, which allows us to directly obtain the cache content in the Buffer when repeatedly reading the RequestBody in a request, and the Buffer itself is a MemoryStream. Of course, we can also implement a set of logic to replace the Body, as long as we rewrite the Stream to support resetting the reading position.
The above is the author’s understanding of how to operate in Request.Body a better way . Regarding the content of the explanation, the author knows that his ability is limited, and the understanding may not be thorough, or even the understanding may not be correct. I hope everyone can understand and welcome everyone. communicate with.