Changing the response in ActionFilter in ASP.Net Core Web Api

I am using both normal data annotation validations and my custom validations in my ASP.Net Core Web Api project. Framework itself adds validation errors to ModelState and according to the business logic, I also add my custom validations using ModelState.AddModelError(). What I want to do is, just before sending the response to the client, I want to check if ModelState.IsValid and if it is not valid, I want to clear the response and send an error JSON in a format I want.

I can do this easily in the controller itself but if I check the ModelState in every controller action, there will be lots of duplicate code. So I wanted to do it in a ActionFilter.

Here is the method I want to use:

public override void OnActionExecuted(ActionExecutedContext context){    if (!context.ModelState.IsValid)    {        // Change the response    }}

I did everything I needed for the first part so I can debug the code until it gets to this method. I also have access to ModelState so I only need to modify the response and send a JSON response. This sounded easy when I thought about it but even after trying more than 10 different codes for changing the response, I haven't been able to do it.

I thought simple code like this would work, but it doesn't (this isn't JSON of course, I tried it with simple text, this doesn't work either):

context.HttpContext.Response.Body.Write(Encoding.UTF8.GetBytes("test"));

So, I have 2 questions:

1. Is this a good idea? Is there a better way to do what I am trying to do?
2. If this is a good idea, how can I modify the response in ActionFilter?

Thanks