Fix DictField HTML input returning empty dict for missing fields

[veeceey]

Summary

Fixes #6234

parse_html_dict always returned an empty MultiValueDict when no matching keys were found in the HTML form input, making it impossible to distinguish between:

• A field that was not specified in the form data
• A field that was specified but had no values

This caused DictField to treat missing fields as if they contained an empty dict, breaking required, default, and partial update logic for multipart/form-data requests.

Changes

• rest_framework/utils/html.py: Added a default parameter to parse_html_dict (consistent with how parse_html_list already works). Returns default when no matching keys are found in the input.
• rest_framework/fields.py: Updated DictField.get_value to pass default=empty so that missing fields are correctly treated as not provided. Also updated DictField.to_internal_value to pass default=data so inner MultiValueDict data is preserved when no dot-separated sub-keys are found.
• rest_framework/serializers.py: Updated Serializer.get_value to use the new default= parameter instead of or empty for consistency.
• tests/test_fields.py: Added 4 new test cases for DictField HTML form input covering:
  • Valid dict input via QueryDict with dot-separated keys
  • Missing field with a default value
  • Missing field that is not required (should be skipped)
  • Missing field that is required (should fail validation)

Test plan

• All existing DictField tests pass
• All 4 new test cases pass
• Full test_fields.py, test_serializer.py, and test_parsers.py suites pass (401 passed, 1 pre-existing failure unrelated to this change)
• Verify in a real Django project with multipart/form-data requests that DictField partial updates work correctly

[veeceey]

Hi maintainers, friendly ping on this PR. It's been open for about 10 days without any review activity. Would really appreciate any feedback or direction when you get a chance. Happy to make adjustments if needed. Thank you!

[browniebroke]

Wouldn't a better default be an empty dict here?

Suggested change

	data = html.parse_html_dict(data, default=data)
	data = html.parse_html_dict(data, default={})

[veeceey]

I tried this but it actually breaks test_querydict_dict_input. When to_internal_value gets called, the data is already a MultiValueDict with parsed keys. parse_html_dict with no prefix looks for dot-prefixed keys which don't match, so default={} would lose the already-parsed data. Keeping default=data as a fallback preserves it correctly. Happy to discuss further though!

[browniebroke]

Looks consistent with what we do for parsing html list 👍🏻 :

#5927

[Copilot]

Pull request overview

This pull request fixes a bug where DictField could not distinguish between a field that was not specified in HTML form data versus a field that was specified but had no values. This caused issues with required fields, default values, and partial updates when using multipart/form-data requests.

Changes:

• Added a default parameter to parse_html_dict() to return a specified value when no matching keys are found (consistent with parse_html_list)
• Updated DictField.get_value() and Serializer.get_value() to pass default=empty so missing fields are correctly detected
• Updated DictField.to_internal_value() to pass default=data to preserve inner MultiValueDict data when no nested keys exist
• Added comprehensive test coverage for DictField HTML form input scenarios

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
rest_framework/utils/html.py	Added default parameter to parse_html_dict() to handle missing fields correctly
rest_framework/fields.py	Updated DictField.get_value() and to_internal_value() to use the new default parameter
rest_framework/serializers.py	Updated Serializer.get_value() to use explicit default=empty parameter for consistency
tests/test_fields.py	Added 4 test cases covering various DictField HTML form input scenarios

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[veeceey]

Thanks for the review! I'll address all three points:

1. Test coverage for serializers.py change - You're right, will add a test that fails on main without the fix to ensure the change to Serializer.get_value() is properly tested.

2. Assert error output - Good catch on the test clarity. I'll update the assert serializer.is_valid() calls to include serializer.errors for better debugging output when tests fail.

3. Default value in to_internal_value - Makes sense to use default={} instead of default=data for cleaner, more consistent behavior.

Pushing the fixes now.

[veeceey]

Hey @browniebroke, pushed the updates:

1. Added a regression test (test_nested_serializer_not_required_with_querydict) that fails on main - exercises the Serializer.get_value path with a QueryDict missing nested fields.

2. Updated all three assert serializer.is_valid() calls to include serializer.errors for better output.

3. Regarding default={} instead of default=data - I tried this but it breaks test_querydict_dict_input. to_internal_value receives a MultiValueDict with already-parsed keys. When parse_html_dict runs with no prefix, it looks for dot-prefixed keys which dont match, so with default={} the parsed data gets lost. The default=data fallback correctly preserves the already-parsed data. Happy to discuss if you see a different approach!

[veeceey]

Hey @browniebroke, circling back on the default={} suggestion — I ended up finding a way to make it work after all. Pushed a commit that uses default={} and falls back to iterating the data items when parse_html_dict returns empty. This way we get the cleaner default you suggested while still preserving the parsed data for the querydict case. Let me know if that looks good or if you'd prefer a different approach!

[browniebroke]

The scope of this PR is doing more than fixing the bug reported in #6234, it is actually improving support for nested objects in HTML forms which was requested a long time ago.

I'm not convinced this is worth doing in terms of risking a regression but will keep this open to revisit in a while

[browniebroke]

That's pretty cool, but feels out of scope of the fix for #6234... This is more related to #2485

We'll have to consider carefully whether this is worth adding

[browniebroke]

I was leaning more towards #9906 because the fix was more targeted, but after more careful review I can see some limits to that other simpler approach, which has some regression.

Although this one seemingly has the same issue, how do one clears a value from a DictField? This test passes on the main branch, but breaks here:

    def test_partial_update_can_clear_html_dict_field(self):
        class TestSerializer(serializers.Serializer):
            field_name = serializers.DictField(required=False)

        serializer = TestSerializer(data=QueryDict('field_name='), partial=True)
        assert serializer.is_valid()
        assert 'field_name' in serializer.validated_data