Add Text To End Of Line Without Loading File

I need to store information into a very big file, in form of many dictionaries. That's not so important, is just to say that I tried to first get all the data into these dictionari

Solution 1:

Did you try any code yourself at all, what were your findings? You might go along the following approach:

withopen('/tmp/bigfile.new', 'w') as output:
    withopen('/tmp/bigfile', 'r') asinput:
        whileTrue:
            line = input.readline().strip()
            ifnot line:
                break
            line += ' Look ma, no hands!'print(line, file=output)

Except of course that instead of 'look ma no hands' you'd have your extra dictionary ;-)

Solution 2:

I will recommed to use pickle to make the process easier. Using pickle there is no need to parse the dict out from the line. And you can do more than add data, you could to update it and remove it too.

import pickle


defupdate_dump_dict(dumps, key, value):
    dict_reborn = pickle.loads(dumps)
    dict_reborn[key] = value
    dumps = pickle.dumps(dict_reborn)
    return dumps


defupdate_line(line_number, key, value):
    withopen('datafile.db', 'wb') as output:
        withopen('new_datafile.db', 'rb') asinput:
            line_number -= 1
            entry = input.readline() 
            if line_number == 0: 
                entry = update_dump_dict(entry, key, value)
            print(entry, file=output)

Example of using pickle:

>>># Dump a dict>>>some_dict = {1: "Hello"}>>>dumped_dict = pickle.dumps(some_dict)>>>print(dumped_dict)
b'\x80\x03}q\x00K\x01X\x05\x00\x00\x00Helloq\x01s.'

>>># Load the dict.>>>dict_reborn = pickle.loads(dumped_dict)>>>print(dict_reborn[1])
Hello

What about human readability?

Well for keep human readability in the file you could use module json:

import json

>>># Dump a dict>>>some_dict = {"key": "Hello"}>>>dumped_dict = json.dumps(some_dict)>>>print(dumped_dict)
{"key": "Hello"}

>>># Load the dict.>>>dict_reborn = json.loads(dumped_dict)>>>print(dict_reborn["key"])
Hello

Of course the previous version has de advantage you don't have to worry about types when you read from file.

Despite of within the recovered dictionary (with json.loads) all will be strings (keys and values) will be always more confortable than parsing the line to extract the data from it.