viennet/scodoc-cohortes
1
0
Fork 0
forked from dubacq/scodoc-cohortes
Code Pull Requests Activity

CLI option handling, uses standard .env, create requirements.txt, update README.

Browse Source
This commit is contained in:
Emmanuel Viennet 2024-10-16 22:44:18 +02:00
parent 3d6d126abc
commit 1cc1b3f496
4 changed files with 138 additions and 97 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.env-example Normal file
View File

@ -0,0 +1,5 @@
# Fichier de configuration accès serveur: à renommer en .env
# et remplir vos valeurs
SCODOC_SERVER=http://localhost:5000 # URL racine, sans slash final
SCODOC_USER=xxx # un login ScoDoc valide et ayant accès à l'API
SCODOC_PASSWORD=xxx

53
README.md
View File

@ -38,6 +38,52 @@ Il existe un dernier type (qui tombe dans le 5 ou le 6 actuellement), c'est
le cas d'un élève en BUT3 qui ne valide pas le BUT3, a validé le BUT2, et
partirait dans une autre filière après. Le cas paraît beaucoup plus douteux que le deuxième type et est pour le moment classé en échec.
## Installation
Créer un virtualenv:
```bash
python3 -m venv venv
source venv/bin/activate
sudo apt-get install libcairo-dev
```
(n'importe quelle version de python récente fera l'affaire).
Puis installer les composants suivants dans ce virtualenv:
```
pip install -r requirements.txt
```
Puis indiquer votre configuration ScoDoc dans le fichier `.env`:
```bash
SCODOC_SERVER=https://votre.serveur.fr
SCODOC_USER=un_utilisateur_api
SCODOC_PASSWORD=son_mot_de_passe
```
### Note pour les développeurs
Pour mettre à jour le fichier `requirements.txt`, lancer (après avoir activé
l'environnement python)
```py
pip freeze > requirements.txt
```
## Usage
Après ouverture du terminal:
```bash
source venv/bin/activate
```
Puis
```bash
./get.py [--techno] [--base 2021] dept ...
```
## FICHIERS
### get.py
@ -48,13 +94,6 @@ Il faut un environnement virtuel pour que soient accessibles les bibliothèques
On peut rajouter l'option `--techno` pour n'avoir que les bacs technos.
### .env
Fichier très important puisqu'il contient toutes les informations de connexion à la base Scodoc.
C'est un fichier CSV (séparateur virgule `,`) avec 4 lignes: username, password, server, baseyear.
baseyear est l'année de départ de la cohorte étudiée.
### redirect.csv
Certains élèves ne reçoivent jamais de décision de jury lorsqu'ils quittent la cohorte, tout en n'étant pas démissionnaires. Ce sont des erreurs administratives, mais il est possible d'indiquer un *résultat de jury* fictif pour ces élèves. La plupart du temps, ce sont des élèves qui abandonnent la formation, et il suffit de leur donner le résultat NAR ou DEM. Dans d'autres cas, ça peut être des élèves en attente de décision parce que le jury n'a pas encore eu lieu, mais on sait déjà quel sera l'issue du jury (par exemple des notes élevés et un stage qui se déroule bien, ou au contraire pas de stage trouvé au mois de septembre).

165
get.py
View File

@ -1,48 +1,82 @@
#!/usr/bin/env python3
# Standard libs
import argparse
import csv
import json
import os
import pdb # used for debugging
import random
import sys
# Third-party libs
import requests
from requests.auth import HTTPBasicAuth
import csv, os, sys
import json
from datetime import datetime
import re
import drawsvg
try:
from dotenv import load_dotenv
except ModuleNotFoundError:
print("\nError: dotenv not installed !", file=sys.stderr)
print("You may install it using:\npip install python-dotenv\n", file=sys.stderr)
def die(msg:str, status=3):
print(msg, file=sys.stderr)
sys.exit(status)
load_dotenv(".env")
SCODOC_SERVER = os.environ.get("SCODOC_URL") or "http://localhost:5000"
SCODOC_USER = os.environ.get("SCODOC_USER") or die("SCODOC_USER must be set in .env or the environment")
SCODOC_PASSWORD = os.environ.get("SCODOC_PASSWORD") or die("SCODOC_PASSWORD must be set in .env or the environment")
API_URL=f"{SCODOC_SERVER}/ScoDoc/api"
# TODO : refactor globals
debug = True # Not used
techno = False # global flag
blocking = True # Die if csv is incorrect
def blockordie():
if blocking:
sys.exit(2)
# Read args from the command line
# then read config from {orderkey}.json
BLOCKING = True # Die if csv is incorrect
# TODO : refactor / put globals in a class, eg Config
depts = []
orderkey = ""
def blockordie(status=2):
if BLOCKING:
sys.exit(status)
class Options:
pass
#
def cli_check():
global techno
global orderkey
"""Read args from the command line
then read config from {orderkey}.json
"""
global orderkey # TODO: globales à supprimer
global depts
index = 1
if sys.argv[index][:2] == "--":
if sys.argv[index] == "--techno":
techno = True
else:
print(f"{sys.argv[0]} [--techno] DEPT [DEPT] ...")
sys.exit(0)
index += 1
orderkey = "_".join(sys.argv[index:])
depts = sys.argv[index:]
parser = argparse.ArgumentParser(description='Process some departments.')
parser.add_argument('--techno', action='store_true', help='Enable TECHNO mode')
parser.add_argument('depts', nargs='+', help='List of departments')
parser.add_argument('--base', '-b', type=int, choices=range(2000, 2667), default=2021,
help='base year for the cohort (integer between 2000 and 2666)')
args = parser.parse_args()
Options.base_year = args.base
Options.techno = args.techno
depts = args.depts
orderkey = "_".join(depts)
if len(depts) == 0:
print(f"{sys.argv[0]} [--techno] DEPT [DEPT] ...")
parser.print_help()
sys.exit(0)
def api_url(dept:str|None=None):
"""L'URL de l'API départementale.
"""
# peut être modifié ici pour n'utiliser que l'API globale
return f"{SCODOC_SERVER}/ScoDoc/{dept}/api" if dept else f"{SCODOC_SERVER}/ScoDoc/api"
cli_check()
@ -62,9 +96,8 @@ def write_conf(key, obj):
conf = read_conf(orderkey)
# Manage default values
def conf_value(xkey: str):
"""Manage default values"""
defaults = {
"spacing": 14,
"thickness": 6,
@ -93,48 +126,9 @@ def conf_value(xkey: str):
return {}
# READ username, password, server, baseyear from secrets file
# This file should be kept really safe
# Currently, only baseyear = 2021 has been tested
server, username, password, baseyear = "", "", "", 0
def read_secrets(filename):
keys = ["server", "username", "password", "baseyear"]
integers = ["baseyear"]
if os.path.exists(filename):
with open(filename, newline="") as csvfile:
csvreader = csv.reader(csvfile, delimiter=",", quotechar='"')
found = 0
for row in csvreader:
for p, k in enumerate(keys):
if row[0] == k:
if k in integers:
globals()[k] = int(row[1])
else:
globals()[k] = row[1]
found |= 0x1 << p
if found != 0xF:
print(f'Des paramètres manquent dans "{filename}" ({found}).')
for p, k in enumerate(keys):
if found & (0x1 << p) == 0:
if k in globals():
g = '"' + globals()[k] + '"'
else:
g = None
print(f"{k} = {g}")
sys.exit(1)
read_secrets(".env")
student = {}
CACHE_FILE = "cache.json"
def load_cache(cache_file):
if os.path.exists(cache_file):
with open(cache_file, "r") as f:
@ -148,14 +142,11 @@ def save_cache(cache, file=None):
with open(CACHE_FILE, "w") as f:
json.dump(cache, f)
cache = load_cache(CACHE_FILE)
# Read color theme
# There are default color values, so may be it should just join the conf.json file
def read_theme():
if os.path.exists("theme.csv"):
with open("theme.csv", newline="") as csvfile:
@ -218,8 +209,8 @@ def get_json(url: str, params=None):
print(f"Requesting {url}")
global token
if token == None:
url_token = f"{server}/api/tokens"
response = requests.post(url_token, auth=HTTPBasicAuth(username, password))
url_token = f"{API_URL}/tokens"
response = requests.post(url_token, auth=HTTPBasicAuth(SCODOC_USER, SCODOC_PASSWORD))
if response.status_code == 200:
token = response.json().get("token")
else:
@ -248,7 +239,7 @@ def get_formsem_from_dept(dept):
cache["formsems"] = {}
if "sem" not in cache:
cache["sem"] = {}
query_url = f"{server}{dept}/api/formsemestres/query"
query_url = f"{api_url(dept)}/formsemestres/query"
formsemestres = get_json(query_url)
result = []
for sem in formsemestres:
@ -262,12 +253,11 @@ def get_formsem_from_dept(dept):
def get_formations_from_dept(dept):
global server
if "formations" in cache and dept in cache["formations"]:
return cache["formations"][dept]
if "formations" not in cache:
cache["formations"] = {}
query_url = f"{server}{dept}/api/formations"
query_url = f"{api_url(dept)}/formations"
formations = get_json(query_url)
result = []
for f in formations:
@ -285,14 +275,14 @@ def get_etuds_from_formsem(dept, semid):
return cache["etudlist"][semid]
if "etudlist" not in cache:
cache["etudlist"] = {}
query_url = f"{server}{dept}/api/formsemestre/{semid}/etudiants/long"
query_url = f"{api_url(dept)}/formsemestre/{semid}/etudiants/long"
result = get_json(query_url)
cache["etudlist"][semid] = result
save_cache(cache)
return result
def get_jury_from_formsem(dept, semid):
def get_jury_from_formsem(dept:str, semid):
if type(semid) == type(0):
semid = str(semid)
if "semjury" in cache and semid in cache["semjury"]:
@ -301,7 +291,7 @@ def get_jury_from_formsem(dept, semid):
cache["semjury"] = {}
# query_url = f"{server}{dept}/Scolarite/Notes/formsemestre_recapcomplet?formsemestre_id={semid}&mode_jury=1&tabformat=json"
query_url = f"{server}{dept}/api/formsemestre/{semid}/decisions_jury"
query_url = f"{api_url(dept)}/formsemestre/{semid}/decisions_jury"
result = get_json(query_url)
cache["semjury"][semid] = result
save_cache(cache)
@ -420,7 +410,7 @@ def analyse_depts():
year = 1
else:
year = (sem["semestre_id"] + 1) // 2
offset = sem["annee_scolaire"] - baseyear - year + 1
offset = sem["annee_scolaire"] - Options.base_year - year + 1
if offset < 0 and offset > -4:
oldsems.add(str(semid))
oldsemsdept[semid] = dept
@ -470,7 +460,7 @@ def analyse_depts():
bacs.add(studentsummary["bac"])
# We skip non-techno students if we are in techno mode
# If we want a mixed reporting, maybe we should change this
if techno and studentsummary["bac"][:2] != "ST": # TODO: change this
if Options.techno and studentsummary["bac"][:2] != "ST": # TODO: change this
continue
if bucket in studentsummary["cursus"]:
semestreerreur = int(bucket) + 1
@ -771,19 +761,19 @@ for etudid in student.keys():
resultyear = redirects[etudid]
redirs[ddd] += 1
strangecases.append(
f"REDI{lastyear} {server}{ddd}/Scolarite/fiche_etud?etudid={etudid}"
f"REDI{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
)
if resultyear == None:
finaloutput = "?" + etud["nickshort"][lastyear]
unknown[ddd] += 1
strangecases.append(
f"????{lastyear} {server}{ddd}/Scolarite/fiche_etud?etudid={etudid}"
f"????{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
)
elif resultyear in ("RAT", "ATJ"):
finaloutput = "?" + etud["nickshort"][lastyear]
unknown[ddd] += 1
strangecases.append(
f"ATTE{lastyear} {server}{ddd}/Scolarite/fiche_etud?etudid={etudid}"
f"ATTE{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
)
elif resultyear in ("RED", "ABL", "ADSUP"):
finaloutput = "RED " + etud["nickshort"][lastyear]
@ -803,7 +793,7 @@ for etudid in student.keys():
elif resultyear in ("NAR", "DEM", "DEF", "ABAN"):
finaloutput = "FAIL " + etud["nickshort"][lastyear]
failure[ddd] += 1
elif resjury["annee"]["annee_scolaire"] != baseyear + lastyear - 1:
elif resjury["annee"]["annee_scolaire"] != Options.base_year + lastyear - 1:
finaloutput = "RED " + etud["nickshort"][lastyear]
checkred = True
if checkred:
@ -826,7 +816,7 @@ for etudid in student.keys():
yearold = cache["sem"][etud["oldsem"]]["annee_scolaire"]
etud["nickshort"][firstyear - 1] = etud["old"] + " " + str(yearold)
yy = yearold
delta = firstyear + baseyear - yy - 2
delta = firstyear + Options.base_year - yy - 2
for i in range(delta, firstyear - 1):
etud["nickshort"][i] = etud["nickshort"][firstyear - 1] + "*" * (
firstyear - 1 - i
@ -1122,7 +1112,6 @@ def crossweight(node_position, node_layer, edges):
return w
import random
def genetic_optimize(node_position, node_layer, edges):

8
requirements.txt Normal file
View File

@ -0,0 +1,8 @@
certifi==2024.8.30
charset-normalizer==3.4.0
drawsvg==2.4.0
idna==3.10
pycairo==1.27.0
python-dotenv==1.0.1
requests==2.32.3
urllib3==2.2.3