Laravel Data two-step validation

[rubenvanassche]

Lately, there needs to be more clarity on when and how laravel-data validates things. At the moment, this is how it works:

If you have a data object with custom-defined magic creation methods, nothing is validated, no property names are mapped, and the method is run to create the object. Unless you're passing in a Request, which always will be validated. The catch here is that this validation happens before creating the data object. And thus, things like this are complicated:

class UserData extends Data
{
  public function __construct(
    public string $first_name,
    public string $last_name,
    public string $full_name,
  )
  {
  }

  public static function fromRequest(UserRequest $request): self
  {
    return new self(
      $request->first_name,
      $request->last_name,
      "{$request->first_name} {$request->last_name}",
    );
  }
}

// $request has ``first_name` and `last_name`

// No errors since no validation runs
UserData::fromRequest($request);

// No errors since no validation runs due to an array
UserData::from($request->toArray());

// ValidationException because `full_name` is missing
UserData::from($request);

The solution is adding #[WithoutValidation] to the property, which stops the error but is weird. Another solution could be adding a hook before validating where the request would be cleaned up, but you probably don't want to mess with the Request object.

Almost the same thing is happening with validateAndCreate, which will validate everything passed to it.

// ValidationException because `full_name` is missing
UserData::validateAndCreate($request->toArray());

The reason why we're doing this? Putting data in PHP objects can cause exceptions. Putting null in a public string $property will fail hard with all kinds of doom Exceptions and Errors, and this is a scenario we want to avoid. Also, casts run at this point. We cannot put a string into a Carbon object; we must cast it before creating the data object.

That being said, the setup we're having now is complicated, and it would be a lot easier to validate things if they are already in the data object.

Proposal

A two-step validation system. Let's look at the from method. If you haven't defined a magic creation method, the package will do its best to create the object by intelligent mapping and casting properties. In this case, we would still do this and generate validation rules. But these validation rules would be the bare minimum to build a valid PHP object.

For a type, this would mean

• if it's nullable, the nullable rule will be added
• if it's Optional, the sometimes rule will be added
• basic types would have their basic rules like string -> string, int -> number, and so on
• data objects would generate rules recursively
• data collectables would do the same

These rules would then be used to validate the payload created by the pipeline. If it is invalid, we return a ValidationException and don't create the PHP object. If it is valid, we create the data object and run a second validation step where we use all the custom rules, attributes rules, and so on defined on the data object.

This second validation step will only be executed when a request is passed. But there would be a few options to trigger this for other payloads:

• In the laravel-data.php config file, always_validate_when_using_from, would be false by default
• In the data object, a method will be added allwaysValidateWhenUsingFrom(): ?bool would be null by default so that if not defined, it doesn't decide if we're validating or not
• A new method fromWithValidation(mixed ...$payloads) which always validate

The reason why we're not validating by default:

• Performance: we sometimes create massive collections of already validated data objects. The extra validation is optional, and inferring all the validation rules is a complicated process that adds a penalty.
• Visibility: when manually writing rules, you know what's happening. An exist in a collection of values is a no-go! But since each data object makes its own rules, we don't have this visibility which is kinda dangerous.

So by default, only request validation but a manual opt-in for always enabling validation.

Trade-offs and inner working

Casts

Currently, casts run after validation, so they know their data is valid. This would not be the case after this change since casts would have to run after the first validation step but before the object creation and, thus second validation step, which usually validates the data before it is passed to the cast.

As a solution, we could let casts define rules that would run in the first validation step. Then, for example the DateTimeInterfaceCast would be able to check if the passed value is a string and a date.

Custom rules per data object

These would run in the second validation step, which means an opportunity. Since the data object already exists, we can pass it through.

Validation attributes

This is a tricky one. Some validation attributes will break since they must work on more complex types. For example, a #[Date] rule will not work on a string but on a Carbon or DateTime object with the new implementation. Luckily the Laravel validator is smart enough, in this case, to still validate the object. But there will probably be other cases where we'll have some things breaking.

Pipeline

It would be the same, but the validation pipe would not run anymore.

Mapping of property names + injecting of defaults

It would be the same and still run in the pipeline. Where would the mapping be used within the validation? The second validation step would always use the property name, that's out of question. But the first step would logically use the mapped names when generating validation rules.

Attributes and messages (in the validator)

It should be generated on the initial validation step since you may also want to overwrite these labels or messages. It could be reused further down the line in the second validation step.

ValidateAndCreate method

It would be removed since we now will have fromWithValidation.

GetValidationRules method

We must return rules from the initial + second validation step merged.

Two step tules

We would run validation twice, the first time with basic rules, the second time with more complicated ones. So for new users of the package this might seem odd that the validation could fail twice in a row with different rules.

That being said, the first step would be the most essential rules, just to make PHP not crash and being able to create the object, if you've written your code well this shouldn't fail normally.

Extra's besides two step validation

Laravel precognition

We want to start supporting Laravel Precognition, this should complete two validation steps before being valid.

Validating from non simple values

This is not working richt now:

class ClientData extends Data
{
    public function __construct(
        public string          $name,
        public DocumentData    $document,
    ) {}
}

class DocumentData extends Data
{
    public function __construct(
        public string $type,
    ) {}
}

it('cannot validate nested data', function () {
    ClientData::validateAndCreate([
      'name' => 'Rick',
      'document' => new DocumentData('passport')
    ]);
});

We expect an array but get an DocumentData object, an an ideal world this would work. Implementing this also has a few quirks: #481

Conclusion

This is a first draft of what could be done. I probably forget a lot of edge cases, so feel free to add a comment when you see one. This thread will only be about refactoring validation into this new form, new features can be added to the larvel-data v4 discussion.

I'm not entirely sure this is the way we want to fix this, if we want to fix this. It is a lot of work, and some things will be broken entirely for this change.

If we're doing this, then v4 will be the target.

Let me know what you think!

[ju5t]

I'm not fully grasping all concepts you've shared, but I'm all in for better validation.

Coincidentally I started working on an MVP today, where we're rewriting our API calls with other services. It is not uncommon for some services to require and return 50+ items in a request. To validate them with attributes makes for confusing code.

Maybe there's an alternative for this that I don't know about, but if not and there will be a validation revamp, I would love to see a solution for this too.

[okdewit]

I think one danger with the current implementation is that people not familiar with this library are not immediately aware of the differences between new FooDTO, FooDTO::from() and FooDTO::validateAndCreate() -- They will happily use new, and assume validation is applied because attributes are present.

My initial assumption from the documentation was that the following would throw an exception:

class FooDTO extends Data
{
    public function __construct(
        #[Max(2)]
        public string $fooString,
    ) {}
}

new FooDTO(fooString: 'too long');

That kind of magic would be hard to achieve I guess, if there's nothing being called to process the attributes.

My personal use case has very little to do with Laravel Form Requests, and more about passing DTOs between domains/contexts in a safe and easily testable/mockable way -- strict/self-validating DTOs can be very useful for that.

I respect that everyone's use cases are different, and I don't really have a strong opinion on solutions. But I do think it could help to add some clear examples to the docs, to explain when validation is applied and when it isn't.

Maybe also allow validate() to be called without arguments ($fooDTO->validate(), instead of $fooDTO->validate($fooDTO->all()))?

My current solution to force strictness is not super elegant, but it works fine:

public function __construct(
    #[Max(2)]
    public string $fooString,
) {
    $this->validate($this->all());
}

[JeroenSchrader]

I think your suggestion does solve an issue I'm currently having.

Goal

Ultimately, I want to use a single Data object for each Model as a replacement for both Request objects and Resource objects.
This basically means I have 2 use-cases:

• Create a new Data object from a Request. I want to validate the data in this case.
• Create a new Data object from an eloquent Model. Not all validation rules should be checked in this case.

Example using the 'Unique' rule

I want to exclude the id of the model present in the request's route parameter from the Unique rule (for example when updating a model).
In the situation below I want to make sure every Project has a unique name within the projects table.
We can use the Unique rule for this:

    [Unique('projects', 'name')]
    public string $name;

However, when updating a Project this rule throws a validation error because the name already exists. We should add an ignore based on the id in the route. We can use a RouteParameterReference class for this:

    [Unique('projects', ignore: new RouteParameterReference('project', 'name'))]
    public string $name;

This is fine when updating the model, the project route parameter is present, but this does not work when creating a new Project.

In this case we created a custom Attribute that only applies the ignore if the request is a PUT request.

    [UniqueOnPut('projects', ignore: new RouteParameterReference('project', 'name'))]
    public string $name;

This now works fine for both creating and updating a Project, but it does not solve the following:

Issue

The RouteParameterReference does not work when creating the Data object via another route where the route parameter is not present.

For example:

   // ProjectData.php
    class ProjectData extends Data
    {
        public function __construct(
            [UniqueOnPut('projects', ignore: new RouteParameterReference('project', 'name'))]
            public string $name;
        ) {
        }
    }

    // CustomerData.php
    class CustomerData extends Data
    {
        public function __construct(
            public string $title,
            public ProjectData $project,
        ) {
        }
    }

    // CustomerController.php
    // PUT /api/customers/1 -> where the route definition is something like /api/customers/{customer}. Note, no 'project' route parameter present.
    public function updateCustomer(CustomerData $data, Customer $customer)
    {
        .....
    }

In the above example the RouteParameterReference throws a CannotResolveRouteParameterReference error because no project route parameter is found.

Conclusion

The new two-step validation does seem to solve the issue. We could then only create the RouteParameterReference when the Data object is being created from a Request object.

The issue regarding RouteParameterReference objects is also discussed here: #297 (comment)

[ronaregen]

hi @JeroenSchrader I have another solution in spatie laravel-data version 4 for unique validation. we can use nullable option to prevent throw an error just like this:
[Unique('projects', ignore: new RouteParameterReference('project', 'name',true))]