Optional Introspect Utility

Creating a utility file to introspect your database is optional. This introspective functionality is designed to accommodate alterations in column structures and the addition of new tables. However, it's crucial to note that deleting a table essential to your code will result in the application breaking.

Create Utility file

Create a new python file 'utils.py' to handle introspection.

add the following code

'''Utilities module'''

import requests
import auth
import json
import os

PBAC_URL = "https://api.devii.io/roles_pbac"
OLD_DICT_FILE = 'old_dict.json' # This file will be used to store the tables and column data prior to introspection

info

When running the introspection for the first time the 'OLD_DICT_FILE' will be created and will report all of the current tables and columns as new.

Next, we will create functions that will load and save the OLD_DICT_FILE in preparation for comparison to changes after introspection is completed.

def load_old_dict():
    if os.path.exists(OLD_DICT_FILE):
        with open(OLD_DICT_FILE, "r") as file:
            return json.load(file)
    else:
        return {}


def save_dict(old_dict):
    with open(OLD_DICT_FILE, 'w') as file:
        json.dump(old_dict, file)

The following functions will do the actual comparison between the data prior to introspection and after introspection.

def compare(old_dict, new_dict):
    changes_detected = False
    message = ''

    # Check for added and removed tables: 
    # to detect new they must be whitelisted first and deleted tables must be removed from whitelist.
    added_tables = set(new_dict.keys()) - set(old_dict.keys())
    removed_tables = set(old_dict.keys()) - set(new_dict.keys())

    if added_tables or removed_tables:
        changes_detected = True
        if added_tables:
            message += f"Added tables: {added_tables}\n"
            for added_table in added_tables:
                message += f"  Fields in {added_table}: {new_dict[added_table]}\n"
        if removed_tables:
            message += f"Removed tables: {removed_tables}\n"


    for tabname, old_fields in old_dict.items():
        new_fields = new_dict.get(tabname, [])
        added_fields = set(new_fields) - set(old_fields)
        removed_fields = set(old_fields) - set(new_fields)

        if added_fields or removed_fields:
            changes_detected = True
            message += f"Changes in {tabname}: \n"
            if added_fields:
                message += f" Added fields: {added_fields}"
            if removed_fields:
                message += f" Removed fields: {removed_fields}"


    if not changes_detected:
        message += 'No changes detected'

    return message

Finally, the Devii introspection function.

def devii_introspect():
    old_dict = load_old_dict()

    introspect = """{
        Utility {introspect}
        }
    """

    util_introspect = {"query": introspect, "variables": {}}
    response = requests.post(PBAC_URL, headers=auth.headers, json=util_introspect)

    data = response.json()["data"]["Utility"]["introspect"]['json']['__schema']

    qtype = [t for t in data['types'] if t['name']=='Query'][0]

    tabnames = [field['name']for field in qtype['fields'] if field['name'] != 'Aggregates']

    new_dict = {}
    for tabname in tabnames:
        fnames = next((x for x in data['types'] if x['name'] == tabname), None)
        if fnames:
            cnames = [field['name'] for field in fnames['fields']]
            new_dict[tabname] = cnames

    message = compare(old_dict, new_dict)

    # Clear and update the old dictionary with new data
    old_dict.clear()
    old_dict.update(new_dict)

    # Save the updated old dictionary to the file
    save_dict(old_dict)

    return message

devii_introspect()

Updates to other files in ToDo app

Note: all the functions outlined below have been included in the project and are outlined to facilitate removal if you would like.

app.py

The following function was added to the app.py file.

#The "GET" method is used here to ensure a response from GraphQL
@app.route("/introspection", methods=["GET"])
def introspect():
    try:
    # Run the devii_introspect function and get messages
        messages = utils.devii_introspect()
        
        # You can return additional data if needed
        result = {"status": "success", "messages": messages}
    except Exception as e:
        # Handle exceptions if introspection fails
        result = {"status": "error", "message": str(e)}
    #to see the result in the console you can uncomment this
    #print ("flask result: ", messages)
    return result

home.html

The following code before </body> at the end of the file.

  <div class="introspect-button">
        <button id="introspectionBtn" class="btn" >Run Introspection</button>
    </div>

script.js

The following has been added at the end of the script.js file.

  introspectionBtn.addEventListener("click", () => {
    fetch("/introspection")
      .then(response => response.json())
      .then(data =>{alert(data.messages);
        window.location.href = "/";
      })
  });