`nbdev` Example function.

Prodb API

Create pandas dataframe for manipulation

1.0 Generate Db

Initialise database with generate_db call.

generate_db[source]

generate_db(dbpath='db.csv', cols=['_c1', '_c2', '_c3'], verbose=True, prefill=False)

Create example database

Generate empty db object with arbitary columns.

df = generate_db(cols=['c1', 'c2', 'c3'])
df.head()

✓💾 db.csv (9 kb)
c1 c2 c3

Generate db with some example data.

df = generate_db(prefill=True)
df.head()

✓💾 db.csv (96 kb)
name mood message time_utc
0 Sam 😊 hi 2021-12-30 15:43:28
1 Grant 😵 hello 2021-12-30 15:43:28

2.0 Insert Row

insert_row[source]

insert_row(df, data, dbpath='db.csv')

insert_rows[source]

insert_rows(df, data, dbpath='db.csv')

Example: Insert single row 

df = generate_db(cols='name mood message'.split())

# ================================================================ #

data = {'name':'George', 'mood': '👹', 'message':'hi'}
df = insert_row(df, data)

# ================================================================ #

display(df)

✓💾 db.csv (18 kb)
name mood message
0 George 👹 hi

Example: Insert multiple rows

Note: prodb allows additional columns to be added after the database has been initialised.

i.e. the ['name', 'mood', 'message'] columns that initialise the db, play nice when the additional time_utc dict is passed in. NaNs are imputed for previous rows.

First, a helper function to clean up timestamp calls.

utc_now[source]

utc_now()

df = generate_db(cols='name mood message'.split())

# ================================================================ #

data = {'name': ['Sam', 'Grant'],
        'mood': ['😊', '😵'],
        'message': ['hello from London, UK', 'hello from Christchurch, NZ'],
        'time_utc' : [utc_now(), utc_now()]}

df = insert_rows(df, data)

# ================================================================ #

display(df)

✓💾 db.csv (18 kb)
name mood message time_utc
0 Sam 😊 hello from London, UK 2021-12-30 15:43:34
1 Grant 😵 hello from Christchurch, NZ 2021-12-30 15:43:34

Add another entry to the bottom of the database using insert_row().

Note: Data columns from dictionary do not need to be in pre-defined order.

data = {'time_utc' : utc_now(),
        'name': 'Luke', 
        'mood': '👹', 
        'message': 'hello from London, UK'}
df = insert_row(df, data)

display(df)
name mood message time_utc
0 Sam 😊 hello from London, UK 2021-12-30 15:43:34
1 Grant 😵 hello from Christchurch, NZ 2021-12-30 15:43:34
2 Luke 👹 hello from London, UK 2021-12-30 15:43:36 
data = {'name':'Bill', 'mood': '👹', 'message':'hi', 'time_utc':arrow.utcnow().format('YYYY-MM-DD HH:mm:ss')}
df = insert_row(df, data)
display(df)
name mood message time_utc
0 Sam 😊 hello from London, UK 2021-12-30 15:43:34
1 Grant 😵 hello from Christchurch, NZ 2021-12-30 15:43:34
2 Luke 👹 hello from London, UK 2021-12-30 15:43:36
3 Bill 👹 hi 2021-12-30 15:43:38

readable_df[source]

readable_df(df, max_rows=8, col_name='human_time')

data = {'name':'Luke', 
        'mood': '😊', 
        'message': 'hello, from UK',
        'time_utc': utc_now()}
df = insert_row(df, data)
readable_df(df, max_rows=10)
name mood message time_utc human_time
0 Sam 😊 hello from London, UK 2021-12-30 15:43:34 10 seconds ago
1 Grant 😵 hello from Christchurch, NZ 2021-12-30 15:43:34 10 seconds ago
2 Luke 👹 hello from London, UK 2021-12-30 15:43:36 just now
3 Bill 👹 hi 2021-12-30 15:43:38 just now
4 Luke 😊 hello, from UK 2021-12-30 15:43:44 just now

Classify

Extending pandas dataframes with our custom functions.

class Prodb[source]

Prodb(data=None, index:Axes | None=None, columns:Axes | None=None, dtype:Dtype | None=None, copy:bool | None=None) :: DataFrame

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Data structure also contains labeled axes (rows and columns). Arithmetic operations align on both row and column labels. Can be thought of as a dict-like container for Series objects. The primary pandas data structure.

Parameters

data : ndarray (structured or homogeneous), Iterable, dict, or DataFrame Dict can contain Series, arrays, constants, dataclass or list-like objects. If data is a dict, column order follows insertion-order.

.. versionchanged:: 0.25.0
   If data is a list of dicts, column order follows insertion-order.

index : Index or array-like Index to use for resulting frame. Will default to RangeIndex if no indexing information part of input data and no index provided. columns : Index or array-like Column labels to use for resulting frame when data does not have them, defaulting to RangeIndex(0, 1, 2, ..., n). If data contains column labels, will perform column selection instead. dtype : dtype, default None Data type to force. Only a single dtype is allowed. If None, infer. copy : bool or None, default None Copy data from inputs. For dict data, the default of None behaves like copy=True. For DataFrame or 2d ndarray input, the default of None behaves like copy=False.

.. versionchanged:: 1.3.0

See Also

DataFrame.from_records : Constructor from tuples, also record arrays. DataFrame.from_dict : From dicts of Series, arrays, or dicts. read_csv : Read a comma-separated values (csv) file into DataFrame. read_table : Read general delimited file into DataFrame. read_clipboard : Read text from clipboard into DataFrame.

Examples

Constructing DataFrame from a dictionary.

d = {'col1': [1, 2], 'col2': [3, 4]} df = pd.DataFrame(data=d) df col1 col2 0 1 3 1 2 4

Notice that the inferred dtype is int64.

df.dtypes col1 int64 col2 int64 dtype: object

To enforce a single dtype:

df = pd.DataFrame(data=d, dtype=np.int8) df.dtypes col1 int8 col2 int8 dtype: object

Constructing DataFrame from numpy ndarray:

df2 = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), ... columns=['a', 'b', 'c']) df2 a b c 0 1 2 3 1 4 5 6 2 7 8 9

Constructing DataFrame from a numpy ndarray that has labeled columns:

data = np.array([(1, 2, 3), (4, 5, 6), (7, 8, 9)], ... dtype=[("a", "i4"), ("b", "i4"), ("c", "i4")]) df3 = pd.DataFrame(data, columns=['c', 'a']) ... df3 c a 0 3 1 1 6 4 2 9 7

Constructing DataFrame from dataclass:

from dataclasses import make_dataclass Point = make_dataclass("Point", [("x", int), ("y", int)]) pd.DataFrame([Point(0, 0), Point(0, 3), Point(2, 3)]) x y 0 0 0 1 0 3 2 2 3

Initialize 

data = {'name': ['Sam', 'Grant'],
        'mood': ['😊', '😵'],
        'message': ['hi', 'hello'],
        'time_utc' : [arrow.utcnow().format('YYYY-MM-DD HH:mm:ss'),
                      arrow.utcnow().format('YYYY-MM-DD HH:mm:ss')]}

df = Prodb(data)

df.prodb_generate(dbpath='db.csv')

✓💾 db.csv (96 KB)	shape: (2, 4)
name mood message time_utc
0 Sam 😊 hi 2021-12-30 16:29:55
1 Grant 😵 hello 2021-12-30 16:29:55

Example: Insert single row 

df = df.prodb_insert({'name':'George', 'mood': '👹', 'message':'hi'})
df.prodb_summary()

✓💾 db.csv (112 KB)	shape: (3, 4)
name mood message time_utc
0 Sam 😊 hi 2021-12-30 16:29:55
1 Grant 😵 hello 2021-12-30 16:29:55
2 George 👹 hi NaN

Example: Insert multiple rows 

data = {'name': ['Multiple', 'Rows'],
        'mood': ['😊', '😵'],
        'message': ['hello', 'hello'],
        'time_utc' : [utc_now(), utc_now()]}

df = df.prodb_insert(data)
df.prodb_summary()

✓💾 db.csv (188 KB)	shape: (5, 4)
name mood message time_utc
0 Sam 😊 hi 2021-12-30 16:29:55
1 Grant 😵 hello 2021-12-30 16:29:55
2 George 👹 hi NaN
3 Multiple 😊 hello 2021-12-30 16:29:56
4 Rows 😵 hello 2021-12-30 16:29:56 
%%time
for i in range(10):
    df = df.prodb_insert(data)
df.prodb_summary()

✓💾 db.csv (948 KB)	shape: (25, 4)
name mood message time_utc
20 Rows 😵 hello 2021-12-30 16:29:56
21 Multiple 😊 hello 2021-12-30 16:29:56
22 Rows 😵 hello 2021-12-30 16:29:56
23 Multiple 😊 hello 2021-12-30 16:29:56
24 Rows 😵 hello 2021-12-30 16:29:56 
CPU times: user 31.2 ms, sys: 0 ns, total: 31.2 ms
Wall time: 66.9 ms