JSON 4: text files
Learning goals
You will learn or practice how to:
- Read and write text files using
FILE*
and the related functions. - Refactor existing code, avoiding breaking previous tests.
Overview
In this homework, you will adapt your code from HW13 to allow reading and writing JSON files. In addition, you will add support for 2 new types: booleans and null. As you add new functionality, you will also be ensuring existing functionality does not break. Refer to the previous parts for specifications on valid and invalid JSON.
JSON is a file format for exchanging hierarchical data between programs, often written
in different programming languages and/or on different computers.
In web programming, this allows a server application written in any language
(e.g., Python or even C) to pass complex structures of information to the
user's browser, which then renders it.
Most often when we work with JSON, it is not through string constants in code, but rather via JSON files,
which have the .json
fle extension.
A JSON file contains text that represents a number, a string, a list, an object, a boolean, or null. However, for the sake of this assignment, you do not need to worry about objects. We will handle them in EC04. Here are some examples:
type | json | data |
---|---|---|
number | 10 | 10 |
string | "ten" | "ten" |
list | [2, 6, 4] | |
boolean | true | true |
null | null |
Linked lists
As in C, whitespace is ignored (except within a string). Thus, the following are all equivalent.
type | json | data |
---|---|---|
list | [2, 6, 4] | |
list | [2, 6, "four"] | |
list | [2, [3, "+", 3], "four"] |
Getting Started on HW15
- The starter code for HW15 is the same as previous JSON assignment. You will be required to modify the header to add the new functions.
- Start by creating a directory and
cp
to copy your code from HW13.
e - Create
print_element_to_file(…)
-
Rename
print_element(…)
toprint_element_to_file(…)
and add the extraFILE*
parameter. Make no other changes yet. -
Recreate
print_element(…)
by having it callprint_element_to_file(…)
. You should only need a single line of code. - Ensure all tests for
print_element(…)
still pass. - Submit.
-
Add tests for
print_element_to_file(…)
. You will need to usefopen(…)
andfclose(…)
. - Ensure your new tests are failing.
- Implement the
FILE*
parameter fromprint_element_to_file(…)
- Ensure both your new test and your previous tests for
print_element(…)
work. - Submit.
- Implement
write_json(…)
.- Implement tests for the function. They can similar to your tests for
print_element_to_file(…)
. - Implement
write_json(…)
. It should callprint_element_to_file(…)
. - Submit.
- Consider if there are any edge cases for the function. If so, add more tests.
- Implement tests for the function. They can similar to your tests for
- Submit.
- Implement
read_json(…)
.- We recommend creating a helper that reads all charaters in a file and returns a malloced string of the contents.
- Declare your helper function, but only implement the minimal code so it compiles.
- Create tests for your helper function.
- Implement enough code in the helper function to pass your first test.
- Ensure tests are passing.
- ...
- Implement the final code for your helper function.
- Ensure all tests are passing for the helper function.
- Add tests for
read_json(…)
. - Implement
read_json(…)
. With the helper function, the implementation should be trivial. - Ensure your tests are passing.
- Submit
- Implement
parse_null(…)
.- Add support for null to the enum and union for
Element
. - Ensure no previous tests broke.
- Add some tests for valid null.
- Implement enough code to pass the tests.
- Submit.
- Add some tests for invalid null.
- Implement enough code to pass the tests.
- Submit.
- Add tests for other JSON functions using null.
- Implement code in other functions for null.
- Ensure all tests are passing, both new and old.
- Add support for null to the enum and union for
- Implement
parse_boolean(…)
.- Add support for booleans to the enum and union for
Element
. - Ensure no previous tests broke.
- Add some tests for valid booleans.
- Recommended: add a helper to allow reusing some logic from
parse_null(…)
. - Implement enough code to pass the tests.
- Submit.
- Add some tests for invalid booleans.
- Implement enough code to pass the tests.
- Submit.
- Add tests for other JSON functions using booleans.
- Implement code in other functions for booleans.
- Ensure all tests are passing, both new and old.
- Add support for booleans to the enum and union for
- Test that there are no memory leaks, even for incorrect input.
264get HW15
to fetch the starter code.
you@ecegrid-thin1 ~/
$
cd 264
you@ecegrid-thin1 ~/264/
$
mkdir hw15
you@ecegrid-thin1 ~/264/
$
cp hw13/* -v -t hw15/
you@ecegrid-thin1 ~/264/
$
cd hw15
you@ecegrid-thin1 ~/264/hw15
$
How much work is this?
You must be disciplined in your development appraoch. If you try to build this all at once, success is extremely unlikely.
Instructor's solution for
parse_null(…)
and parse_bool(…)
is 15 sloc
(5 sloc in the body of a helper called by both).
Instructor's solution for
write_json(…)
and read_json(…)
is 27 sloc
(in addition to the sloc from the body of print_element_to_file(…)
).
Changes to existing code in JSON is 13 sloc.
SLOC means “source lines of code” and does not lines that are blank or contain only comments. The figures above do not count test_json.c. Also, those figures are based on a tight—but defensible—solution that meets code quality standards and uses no methods or language features we have not yet covered.
#}Requirements
- Your submission must contain each of the following files, as specified:
file contents json.h type definitions Element- Everything that was in element before.
- To the type enum, add two new types:
ELEMENT_NULL
andELEMENT_BOOL
. - To the struct, add two new fields:
void* as_null
andbool as_bool
.
function declarations one for each required function in your json.c - Do not include helpers (if any) here.
json.c functions All functions that were required in json.c previously. Only new functions or functions that were changed will be listed below. parse bool(bool✶ a value, char const✶✶ a pos)
→ return type: boolSet*a_value
to whatever boolean value is found at*a_pos
.*a_pos
is initially the address of the first character of the boolean literal in the input string.*a_value
is the (already allocated) location where the parsed boolean should be stored.- Return
true
if a properly formed boolean literal was found at*a_pos
.*a_pos
should refer to the next character in the input string, i.e., after the last character of the boolean literal. Note that trailing characters are acceptable, just like previous functions. -
- Ex:
parse_bool(…)
should returntrue
for true, false, and falsehood
false
. - Ex:
- Return
false
if boolean literal was not found.*a_pos
should refer to the unexpected character that indicated a problem.- Ex:
parse_bool(…)
should returnfalse
for A, 123, "true", and truly
*a_pos
will be'l'
as that is the first invalid character. - Ex:
- ⚠ Calling
parse_bool(…)
should not result in any calls tomalloc(…)
. - ⚠ Do not confuse the boolean return value from the result. The return value determines whether a boolean was found, while
*a_value
is the boolean that was found. - Whenever
parse_bool(…)
returnsfalse
,*a_value
should not be modified.
parse null(char const✶✶ a pos)
→ return type: boolDetermine whethernull
is found at*a_pos
.*a_pos
is initially the address of the first character of the null literal in the input string.- Return
true
if a properly formed null literal was found at*a_pos
.*a_pos
should refer to the next character in the input string, i.e., after the last character of the null literal. Note that trailing characters are acceptable, just like previous functions. -
- Ex:
parse_null(…)
should returntrue
for null, null literal, and nullify
null
. - Ex:
- Return
false
if boolean literal was not found.*a_pos
should refer to the unexpected character that indicated a problem.- Ex:
parse_null(…)
should returnfalse
for A, 123, "null", and nil
*a_pos
will be'i'
as that is the first invalid character. - Ex:
- ⚠ Calling
parse_null(…)
should not result in any calls tomalloc(…)
. - This function has no result, as the only possible result is
null
.
parse element(Element✶ a element, char const✶✶ a pos)
→ return type: bool- First, eat any whitespace at
*a_pos
.- “Eat whitespace” just means to skip over any whitespace characters (i.e., increment
*a_pos
untilisspace(**a_pos)==false
).
- “Eat whitespace” just means to skip over any whitespace characters (i.e., increment
- Next, decide what kind of element this is.
- If it's a digit (
isdigit(**a_pos)
) or hyphen ('-'
), set the element'stype
toELEMENT_INT
and callparse int(&(a element -> as int), a pos)
. - If it's a string (
**a_pos=='"'
), then set the element'stype
toELEMENT_STRING
and callparse string(&(a element -> as string), a pos)
. - If it's a list (
**a_pos == '['
), then set the element'stype
toELEMENT_LIST
and call:parse list(&(a element -> as list), a pos)
. - If it's a boolean (
**a_pos == 't'
or**a_pos == 'f'
), then set the element'stype
toELEMENT_BOOL
and call:parse bool(&(a element -> as bool), a pos)
. - If it's null (
**a_pos == 'n'
), then set the element'stype
toELEMENT_NULL
, set the element'sas_null
toNULL
, and call:parse null(a pos)
.
- If it's a digit (
- Return whatever was returned by the relevant function.
- If none of those functions was called—i.e., if the next character was none of the expected characters, then return
false
.
- If none of those functions was called—i.e., if the next character was none of the expected characters, then return
- Do not modify
*a_pos
directly inparse_element(…)
, except for eating whitespace.*a_pos
can—and should—be modified in the relevant parse functions.
- Caller is responsible for freeing memory by calling
free_element(…)
wheneverparse_element(…)
returnstrue
. - Whenever
parse_element(…)
returnsfalse
, do not modify*a_element
, and free any heap memory that was allocated prior to discovery of the error.
print element to file(Element element, FILE✶ file)
→ return type: voidGiven anElement
object, print it in JSON notation to the passed file.- Spacing is up to you, as long as it is valid JSON.
- If element is an integer, print it using
fprintf(…)
. - If element is a string, then print it (with double-quotes) using
fprintf(…)
. - If element is a list, print a
'['
. Then print each element in the list usingprint_element_to_file(…)
(recursively), separated by commas. Finally, print']'
. - If element is a boolean or null, print it using a string constant.
print element(Element element)
→ return type: voidThis function should be exactly one line. It should simply callprint_element_to_file(…)
with the proper argument for printing to the console.free element(Element element)
→ return type: voidFree the contents of theElement
, as needed. Note that this is the same description from before, just copied over to emphasize that booleans and null do not need to be freed.- If it contains a string, free the string.
- If it contains a linked list, free the list, including all elements.
- ⚠ Do not attempt to free the
Element
object itself.free_element(element)
only frees dynamic memory that element refers to.
write json(char const✶ filename, Element element)
→ return type: voidWrites a JSON element to the file defined byfilename
.- Opens the file named
filename
. - Create the file if it does not exist, replace contents if it does exist.
- Write the contents of
element
to the file. - Hint: use
print_element_to_file(…)
. - ⚠ Do not add the
.json
extension automatically. The caller will pass in the full filename they wish to create. If the caller passes in"example"
, create a file namedexample
, not.example.json
read json(char const✶ filename, Element✶ a element)
→ return type: boolReads a JSON element from the file defined byfilename
.- Opens the file named
filename
and reads in the contents as a JSON element. - Returns
true
if the contents are valid JSON, and the file contains no other text (other than whitespace). - Returns false and prints an appropiate error message to
stderr
if:- The file does not exist.
- The file does not contain a valid JSON element.
- The file starts with a valid JSON element but has non-whitespace trailing characters.
- Caller is responsible for freeing memory
in the returned element by calling
free_element(…)
wheneverread_json(…)
returnstrue
. - Whenever
read_json(…)
returnsfalse
, do not modify*a_element
, and free any heap memory that was allocated prior to discovery of the error. - You are free to use
malloc(…)
as needed for parsing the element, as long as you have no memory leaks. The caller is only responsible for freeing the element (if valid), any other memory you allocate is the responsiblity ofread_json(…)
to free. - Hint: you should use
parse_element(…)
to parse the JSON rather than duplicating logic. We talked in lecture about how to read all text from a file into a string. - ⚠ The last requirement for making a file valid JSON is different from
the her parse functions. It is possible that
read_json(…)
will returntrue
for a string, and the same string as a file will leadread_json(…)
to returnfalse
. - ⚠ Do not add the
.json
extension automatically. The caller will pass in the full filename they wish to create. If the caller passes in"example"
, create a file namedexample
, not.example.json
test_json.c functions main(int argc, char✶ argv[])
→ return type: intTest your all of the above functions using yourminiunit.h.
.- This should consist primarily of calls to
mu_run(_test_▒▒▒)
. - 100% code coverage is required.
- Your main(…) must return EXIT_SUCCESS.
- You may ignore any trailing characters in the input string in any function starting
with
parse_
, as long as the input starts with a well-formed JSON element.- Acceptable: 123AAA, "12"AAA, "12",[,
read_json(…)
, other than whitespace. - You only need to support the specific features of JSON that are explicitly required in this assignment description. You do not need to support unicode (e.g., "萬國碼", "يونيكود", "യൂണികോഡ്"), objects/dictionaries (e.g., {"a":1, "b":2}), backslash escapes (e.g., "\n"), embedded quotes (e.g., "He said, \"Roar!\""), floating point numbers (e.g., 3.1415), non-decimal notations (e.g., 0xdeadbeef, 0600)
- Do not add helper functions to json.h.
- There may be no memory faults (e.g., leaks, invalid read/write, etc.), even when
parse_▒▒▒(…)
orread_json(…)
return false. -
The following external header files, functions, and symbols are
allowed.
header functions/symbols allowed in… stdbool.h bool
,true
,false
json.c
,test_json.c
stdio.h fprintf
,fputc
,fputs
,stdout
,fflush
,fopen
,fclose
,fseek
,ftell
,rewind
,fgetc
,fread
json.c
,test_json.c
assert.h assert
json.c
,test_json.c
ctype.h isdigit
,isspace
json.c
,test_json.c
stdlib.h EXIT_SUCCESS
,abs
,malloc
,free
,size_t
json.c
,test_json.c
string.h strncpy
,strchr
,strlen
,strcmp
json.c
,test_json.c
limits.h INT_MIN
,INT_MAX
json.c
,test_json.c
miniunit.h anything
test_json.c
clog.h anything
json.c
,test_json.c
- Submissions must meet the code quality standards and the course policies on homework and academic integrity.
Submit
To submit HW15 from within your hw15 directory,
type
264submit HW15 json.c json.h test_json.c miniunit.h clog.h Makefile *.json
If your code does not depend on miniunit.h or clog.h, those may be omitted. Your json.h will most likely be identical to the starter. Makefile will not be checked, but including it may help in case we need to do any troubleshooting. Make sure to submit all required json files required by your tests. It is your responsibility to ensure the tester has all the files required to test. You are encouraged to modify your Makefile to do so.
Pre-tester ●
The pre-tester for HW15 has been released and is ready to use.
Q&A
How can I structure my tests?
Here's a start. (We may add to this at some point.)// OK TO COPY / ADAPT this snippet---but ONLY if you understand it completely. // ⚠ Do not copy blindly. // // This test is nowhere near adequate on its own. It is provided to illustrate how to // use helper functions to streamline your test code. #include <stdio.h> #include <stdlib.h> #include "json.h" #include "miniunit.h" // helper to create a file in a test. You are also free to manually create files // as long as you include them in your submission static void _write_string_to_file(char const* filename, char const* contents) { FILE* file = fopen(filename, "w"); fputs(contents, file); fclose(file); } static char* _read_file_to_string(char const* filename) { // TODO: will implement this in class, and add example back here after class } // all tests from JSON 3, do not delete old tests!!! static int _test_read_int_zero() { mu_start(); //────────────────────────────────────────────────────── // create a text file with the contents 0 char const* filename = "_test_read_int_zero.json"; _write_string_to_file(filename, "0"); // test the read function Element result; // will be initialized in read_json(…) bool is_success = read_json(filename, &result); mu_check(is_success); // because the input is valid mu_check(result.type == ELEMENT_INT); mu_check(result.as_int == 0); free_element(element); //────────────────────────────────────────────────────── mu_end(); } static int _test_write_int_zero() { mu_start(); //────────────────────────────────────────────────────── Element element = { .type = ELEMENT_INT, .as_int = 0 }; // create the JSON using our logic const char* filename = "_test_write_json_int_zero.json"; write_json(filename, element); free_element(element); // read the file in to validate its contents char* contents = _read_file_to_string(filename); mu_check_strings_equal(contents, "0"); free(contents); //────────────────────────────────────────────────────── mu_end(); } int main(int argc, char* argv[]) { mu_run(_test_read_int_zero); mu_run(_test_write_int_zero); return EXIT_SUCCESS; }
That's a lot of duplication! Can we make our tests more concise?
You could create helper functions to make your tests easier to write. Some examples are below, but do not feel limited to these examples. You may copy/adapt this code—but only if you understand it completely. ⚠ Do not copy blindly.// FANCY way of testing, using a helper function that returns a mu_check result. // // Okay to copy/adapt, but ONLY IF YOU UNDERSTAND THIS CODE COMPLETELY. // ⚠ Do not copy blindly. #include <stdio.h> #include <stdlib.h> #include <string.h> #include "json.h" #include "miniunit.h" // other helpers static bool _compare_elements(Element a, Element b) { // check element types match, check values match // will be recursive for lists // feel free to print additional debug information about what did not match, // this function is to help you find bugs } static void _write_string_to_file(char const* filename, char const* contents) { FILE* file = fopen(filename, "w"); fputs(contents, file); fclose(file); } // functions to generate tests static int _make_valid_read_test(char const* filename, char const* jsonString, Element expected) { mu_start(); _write_string_to_file(filename, jsonString); Element actual; bool is_success = read_json(filename, actual); mu_check(is_success); mu_check(_compare_elements(expected, actual)); free_element(actual); // you can leave out freeing expected, if you ensure expected is only stack allocated data // its possible to stack allocate a linked list, just a bit more awkward free_element(expected); mu_end(); } static int _make_invalid_read_test(char const* filename, char const* jsonString) { mu_start(); _write_string_to_file(filename, jsonString); Element actual; bool is_success = read_json(filename, actual); mu_check(!is_success); mu_end(); } // actual tests static int _test_read_int_zero() { return _make_valid_read_test("_test_read_int_zero.json", "0", (Element){ .type = ELEMENT_INT, .value = 0 }); } static int _test_read_string_multicharacter() { return _make_valid_read_test("_test_read_string_multicharacter.json", ""hello"", (Element){ .type = ELEMENT_STRING, .value = _copy_string("hello") // heap allocate so free_element works }); } static int _test_read_invalid_trailing() { return _make_invalid_read_test("_test_read_invalid_trailing.json", "-123ABC"); } int main(int argc, char* argv[]) { mu_run(_test_read_int_zero); mu_run(_test_read_string_multicharacter); mu_run(_test_read_invalid_trailing); return EXIT_SUCCESS; }
⚠ If you do not understand this code, do not use it.The code above covers just the
read_json(…)
part of this assignment, but you could create something similar for other functions.Why does
read_json(…)
not allow trailing characters?Most of theparse
functions support trailing characters to make it easier to implement nested JSON structures such as lists and objects. Allowing trailing characters means"1, 2]"
is a valid int, allowingparse_list(…)
to simply callparse_element(…)
.Sinceread_json(…)
is not recurive, to be consistent with the JSON spec it disallows trailing character.But how am I supposed to use
With howparse_element(…)
inread_json(…)
? it supports trailing characters!pos
works, its trivial to check if we reached the end of a string fromparse_element(…)
. If you are stuck, think about how we have been testing it with miniunit.What is "an appropiate error message" for
Print a message that describes the reason you returned false, there are no strict requirements. As an example, suppose you encounter an invalid element, you could print something like:read_json(…)
returning false?Unexpected %c at position %d in JSON file %s.
Usingpos
and the full file contents to determine the character that caused the error and its position.What is
stderr
? How do I print to it?When writing output to the console, we can actually write to eitherstdout
orstderr
. Both are redirected to the console by default. But you can for example redirectstdout
to one file andstderr
to a second file. If you understandstdout
, usingstderr
should be trivial.However, do not get hung up onstderr
. If you cannot figure it out, print errors tostdout
.
Updates
7/29/2022 |
|