forked from dubacq/scodoc-cohortes
Compare commits
3 Commits
Author | SHA1 | Date | |
---|---|---|---|
|
67bab0af54 | ||
|
6ccb348361 | ||
|
41bf7fd069 |
56
README.md
56
README.md
@ -1,9 +1,8 @@
|
||||
# scodoc-cohortes
|
||||
|
||||
Visualisation des cohortes de BUT depuis scodoc
|
||||
*Visualisation des cohortes de BUT depuis scodoc*
|
||||
|
||||
Ce programme utilise l'API pour générer des graphes de Sankey permettant
|
||||
de visualiser le devenir d'une cohorte.
|
||||
Ce programme utilise l'API pour générer des graphes de Sankey permettant de visualiser le devenir d'une cohorte.
|
||||
|
||||
La notion de cohorte devient difficile à traiter lorsqu'on considère
|
||||
l'existence de passerelles permettant d'entrer à n'importe quel année
|
||||
@ -38,17 +37,19 @@ Il existe un dernier type (qui tombe dans le 5 ou le 6 actuellement), c'est
|
||||
le cas d'un élève en BUT3 qui ne valide pas le BUT3, a validé le BUT2, et
|
||||
partirait dans une autre filière après. Le cas paraît beaucoup plus douteux que le deuxième type et est pour le moment classé en échec.
|
||||
|
||||
## Installation
|
||||
## Installation (Linux et MacOS)
|
||||
|
||||
Créer un virtualenv:
|
||||
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
sudo apt-get install libcairo-dev
|
||||
sudo apt install libcairo-dev
|
||||
```
|
||||
(n'importe quelle version de python récente fera l'affaire).
|
||||
|
||||
Si vous êtes sur un système qui n'est pas de la famille Debian/Ubuntu, il faudra sans doute remplacer la dernière ligne par votre installeur de paquets préféré. Cette opération n'est à faire qu'une seule fois par machine.
|
||||
|
||||
Puis installer les composants suivants dans ce virtualenv:
|
||||
|
||||
```
|
||||
@ -63,16 +64,35 @@ SCODOC_USER=un_utilisateur_api
|
||||
SCODOC_PASSWORD=son_mot_de_passe
|
||||
```
|
||||
|
||||
## Installation (Windows)
|
||||
|
||||
Exécuter un terminal depuis le répertoire où est installé le programme. Les commandes à taper dans l'interpréteur de commandes *PowerShell* sont :
|
||||
|
||||
```powershell
|
||||
python.exe -m venv venv
|
||||
venv\Scripts\Activate.ps1
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
Puis indiquer votre configuration ScoDoc dans le fichier `.env`:
|
||||
|
||||
```bash
|
||||
SCODOC_SERVER=https://votre.serveur.fr
|
||||
SCODOC_USER=un_utilisateur_api
|
||||
SCODOC_PASSWORD=son_mot_de_passe
|
||||
```
|
||||
|
||||
Pycairo ne devrait pas s'installer. Si vous savez installer un pycairo fonctionnel pour Windows, n'hésitez pas à nous proposer de le rajouter à ces instructions.
|
||||
|
||||
### Note pour les développeurs
|
||||
|
||||
Pour mettre à jour le fichier `requirements.txt`, lancer (après avoir activé
|
||||
l'environnement python)
|
||||
```py
|
||||
pip freeze > requirements.txt
|
||||
pip freeze|sed -e '/pycairo/ s/$/; platform_system != "Windows"/g' > requirements.txt
|
||||
```
|
||||
## Usage
|
||||
|
||||
Après ouverture du terminal:
|
||||
Après ouverture du terminal (une seule fois par terminal):
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
@ -88,15 +108,21 @@ Puis
|
||||
|
||||
### get.py
|
||||
|
||||
Le programme principal. Il prend en argument des acronymes de département (par exemple GEA ou INFO) et fabrique un graphe comportant les formations BUT de ce département (ou ces départements dans le même graphe, s'ils sont plusieurs sur la ligne de commande).
|
||||
C'est l'exécutable. Il prend en argument des acronymes de département (par exemple GEA ou INFO) et fabrique un graphe comportant les formations BUT de ce département (ou ces départements dans le même graphe, s'ils sont plusieurs sur la ligne de commande).
|
||||
|
||||
Il faut un environnement virtuel pour que soient accessibles les bibliothèques Python pycairo, drawsvg, requests. A priori libcairo est optionnel, mais le graphe marchera moins bien sans. La bibliothèque (système) `libcairo2` doit aussi être installée (`apt install libcairo-dev` ou équivalent).
|
||||
|
||||
On peut rajouter l'option `--techno` pour n'avoir que les bacs technos.
|
||||
|
||||
Liste complète des options :
|
||||
|
||||
* `--techno` : ne sélectionne que les bacs technos
|
||||
* `--base` *year* : prend la cohorte constituée de tous les élèves qui ont touché le S1 ou le S2 dans l'année *year*, le S3 ou le S4 dans l'année *year+1*, et le S5 ou le S6 dans l'année *year+2*.
|
||||
* *dept1* (acronyme d'un département, suivi éventuellement d'autres acronymes de département) : sélectionne les départements mentionnés. Si aucun département n'est mentionné, donne l'aide.
|
||||
|
||||
### redirect.csv
|
||||
|
||||
Certains élèves ne reçoivent jamais de décision de jury lorsqu'ils quittent la cohorte, tout en n'étant pas démissionnaires. Ce sont des erreurs administratives, mais il est possible d'indiquer un *résultat de jury* fictif pour ces élèves. La plupart du temps, ce sont des élèves qui abandonnent la formation, et il suffit de leur donner le résultat NAR ou DEM. Dans d'autres cas, ça peut être des élèves en attente de décision parce que le jury n'a pas encore eu lieu, mais on sait déjà quel sera l'issue du jury (par exemple des notes élevés et un stage qui se déroule bien, ou au contraire pas de stage trouvé au mois de septembre).
|
||||
Certains élèves ne reçoivent jamais de décision de jury lorsqu'ils quittent la cohorte, tout en n'étant pas démissionnaires. Ce sont des erreurs administratives, mais il est possible d'indiquer un *résultat de jury* **fictif** pour ces élèves. La plupart du temps, ce sont des élèves qui abandonnent la formation, et il suffit de leur donner le résultat NAR ou DEM. Dans d'autres cas, ça peut être des élèves en attente de décision parce que le jury n'a pas encore eu lieu, mais on sait déjà quel sera l'issue du jury (par exemple des notes élevés et un stage qui se déroule bien, ou au contraire pas de stage trouvé au mois de septembre).
|
||||
|
||||
**Format :** format CSV avec virgule comme séparateur. Les lignes vides ou commençant par # sont ignorées.
|
||||
|
||||
@ -107,7 +133,7 @@ Certains élèves ne reçoivent jamais de décision de jury lorsqu'ils quittent
|
||||
|
||||
### theme.csv
|
||||
|
||||
Possibilité de choisir les couleurs pour chacune des catégories.
|
||||
Ce fichier offre la ossibilité de choisir les couleurs pour chacune des catégories.
|
||||
|
||||
**Format :** format CSV avec virgule comme séparateur. Les couleurs sont au format de sankeymatic.com, soit le format hexadécimal d'HTML sauf la catégorie TRANSPARENT qui vaut #FFFFFF.0 (blanc transparent). Les catégories sont `+DUT`, `QUIT`, `SUCCESS`, `NORMAL`, `FAIL`, `OLD`, `NEW`, `TRANSPARENT`, `RED`.
|
||||
|
||||
@ -123,8 +149,16 @@ Possibilité de choisir les couleurs pour chacune des catégories.
|
||||
|
||||
### <dept>.json
|
||||
|
||||
Ce fichier permet de configurer plus finement le graphique pour le département (ou la combinaison de département, séparés par des `_`) indiqué dans son nom de fichier.
|
||||
|
||||
**Format :** format JSON (utiliser un éditeur qui sait repérer les erreurs de format est recommandé).
|
||||
|
||||
**Liste des clés :** à venir.
|
||||
|
||||
### <dept>.svg
|
||||
|
||||
Ce fichier est la sortie du programme. C'est un graphique au format SVG, facile à incorporer dans des pages Web ou autres programmes de traitement de texte.
|
||||
|
||||
### best-<dept>.json
|
||||
|
||||
Ce fichier contient le résultat d'une recherche heuristique pour avoir un graphe meilleur. Il peut être supprimé si le graphe ne s'améliore pas sur des lancements successifs. Il peut aussi être modifié à la main.
|
||||
Ce fichier contient le résultat d'une recherche heuristique pour avoir un graphe visuellement plus satisfaisant. Il peut être supprimé si le graphe ne s'améliore pas par des lancements successifs. Il peut aussi être modifié à la main. C'est essentiellement l'ordre des balises
|
529
get.py
529
get.py
@ -20,17 +20,23 @@ except ModuleNotFoundError:
|
||||
print("\nError: dotenv not installed !", file=sys.stderr)
|
||||
print("You may install it using:\npip install python-dotenv\n", file=sys.stderr)
|
||||
|
||||
def die(msg:str, status=3):
|
||||
|
||||
def die(msg: str, status=3):
|
||||
print(msg, file=sys.stderr)
|
||||
sys.exit(status)
|
||||
|
||||
|
||||
load_dotenv(".env")
|
||||
|
||||
SCODOC_SERVER = os.environ.get("SCODOC_URL") or "http://localhost:5000"
|
||||
SCODOC_USER = os.environ.get("SCODOC_USER") or die("SCODOC_USER must be set in .env or the environment")
|
||||
SCODOC_PASSWORD = os.environ.get("SCODOC_PASSWORD") or die("SCODOC_PASSWORD must be set in .env or the environment")
|
||||
SCODOC_SERVER = os.environ.get("SCODOC_SERVER") or "http://localhost:5000"
|
||||
SCODOC_USER = os.environ.get("SCODOC_USER") or die(
|
||||
"SCODOC_USER must be set in .env or the environment"
|
||||
)
|
||||
SCODOC_PASSWORD = os.environ.get("SCODOC_PASSWORD") or die(
|
||||
"SCODOC_PASSWORD must be set in .env or the environment"
|
||||
)
|
||||
|
||||
API_URL=f"{SCODOC_SERVER}/ScoDoc/api"
|
||||
API_URL = f"{SCODOC_SERVER}/ScoDoc/api"
|
||||
|
||||
# TODO : refactor globals
|
||||
debug = True # Not used
|
||||
@ -40,12 +46,16 @@ BLOCKING = True # Die if csv is incorrect
|
||||
depts = []
|
||||
orderkey = ""
|
||||
|
||||
|
||||
def blockordie(status=2):
|
||||
if BLOCKING:
|
||||
sys.exit(status)
|
||||
|
||||
|
||||
class Options:
|
||||
pass
|
||||
|
||||
|
||||
#
|
||||
def cli_check():
|
||||
"""Read args from the command line
|
||||
@ -54,11 +64,17 @@ def cli_check():
|
||||
global orderkey # TODO: globales à supprimer
|
||||
global depts
|
||||
|
||||
parser = argparse.ArgumentParser(description='Process some departments.')
|
||||
parser.add_argument('--techno', action='store_true', help='Enable TECHNO mode')
|
||||
parser.add_argument('depts', nargs='+', help='List of departments')
|
||||
parser.add_argument('--base', '-b', type=int, choices=range(2000, 2667), default=2021,
|
||||
help='base year for the cohort (integer between 2000 and 2666)')
|
||||
parser = argparse.ArgumentParser(description="Process some departments.")
|
||||
parser.add_argument("--techno", action="store_true", help="Enable TECHNO mode")
|
||||
parser.add_argument("depts", nargs="+", help="List of departments")
|
||||
parser.add_argument(
|
||||
"--base",
|
||||
"-b",
|
||||
type=int,
|
||||
choices=range(2000, 2667),
|
||||
default=2021,
|
||||
help="base year for the cohort (integer between 2000 and 2666)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
@ -71,11 +87,13 @@ def cli_check():
|
||||
parser.print_help()
|
||||
sys.exit(0)
|
||||
|
||||
def api_url(dept:str|None=None):
|
||||
"""L'URL de l'API départementale.
|
||||
"""
|
||||
|
||||
def api_url(dept: str | None = None):
|
||||
"""L'URL de l'API départementale."""
|
||||
# peut être modifié ici pour n'utiliser que l'API globale
|
||||
return f"{SCODOC_SERVER}/ScoDoc/{dept}/api" if dept else f"{SCODOC_SERVER}/ScoDoc/api"
|
||||
return (
|
||||
f"{SCODOC_SERVER}/ScoDoc/{dept}/api" if dept else f"{SCODOC_SERVER}/ScoDoc/api"
|
||||
)
|
||||
|
||||
|
||||
cli_check()
|
||||
@ -96,6 +114,7 @@ def write_conf(key, obj):
|
||||
|
||||
conf = read_conf(orderkey)
|
||||
|
||||
|
||||
def conf_value(xkey: str):
|
||||
"""Manage default values"""
|
||||
defaults = {
|
||||
@ -104,10 +123,10 @@ def conf_value(xkey: str):
|
||||
"fontsize_name": 10,
|
||||
"fontsize_count": 14,
|
||||
"width": 1300,
|
||||
"height": 900,
|
||||
"height": 0,
|
||||
"hmargin": 20,
|
||||
"parcours_separator": "/",
|
||||
"year_separator": "",
|
||||
"year_separator": " ",
|
||||
"rank_separator": "",
|
||||
"diplome_separator": "",
|
||||
}
|
||||
@ -117,10 +136,10 @@ def conf_value(xkey: str):
|
||||
return defaults[xkey]
|
||||
if xkey[-9:] == "separator":
|
||||
return " "
|
||||
if xkey == "nick":
|
||||
return "{diplome}{rank}{multidepartment}{modalite}{parcours}"
|
||||
if xkey == "nick" or xkey == "displayname":
|
||||
return "{diplome}{rank}{multidepartment}{modalite}{parcours}{year}"
|
||||
if xkey == "extnick":
|
||||
return "{rank}{multidepartment}{diplomenobut}{modaliteshort}"
|
||||
return "{ext}{rank}{multidepartment}{diplomenobut}{modaliteshort}"
|
||||
if xkey == "orders":
|
||||
return [[], [], [], [], []]
|
||||
return {}
|
||||
@ -129,6 +148,7 @@ def conf_value(xkey: str):
|
||||
student = {}
|
||||
CACHE_FILE = "cache.json"
|
||||
|
||||
|
||||
def load_cache(cache_file):
|
||||
if os.path.exists(cache_file):
|
||||
with open(cache_file, "r") as f:
|
||||
@ -142,6 +162,7 @@ def save_cache(cache, file=None):
|
||||
with open(CACHE_FILE, "w") as f:
|
||||
json.dump(cache, f)
|
||||
|
||||
|
||||
cache = load_cache(CACHE_FILE)
|
||||
|
||||
|
||||
@ -210,7 +231,9 @@ def get_json(url: str, params=None):
|
||||
global token
|
||||
if token == None:
|
||||
url_token = f"{API_URL}/tokens"
|
||||
response = requests.post(url_token, auth=HTTPBasicAuth(SCODOC_USER, SCODOC_PASSWORD))
|
||||
response = requests.post(
|
||||
url_token, auth=HTTPBasicAuth(SCODOC_USER, SCODOC_PASSWORD)
|
||||
)
|
||||
if response.status_code == 200:
|
||||
token = response.json().get("token")
|
||||
else:
|
||||
@ -282,7 +305,7 @@ def get_etuds_from_formsem(dept, semid):
|
||||
return result
|
||||
|
||||
|
||||
def get_jury_from_formsem(dept:str, semid):
|
||||
def get_jury_from_formsem(dept: str, semid):
|
||||
if type(semid) == type(0):
|
||||
semid = str(semid)
|
||||
if "semjury" in cache and semid in cache["semjury"]:
|
||||
@ -311,42 +334,9 @@ def get_override(sem, xkey, default=None):
|
||||
return default
|
||||
|
||||
|
||||
def analyse_student(semobj, etud, year=None):
|
||||
"""Returns the final (department,diplome,rank,modalite,parcours,nickname) tuple from etudid in semid, taking into accounts overrides."""
|
||||
session_id = semobj["session_id"].split("-")
|
||||
department = session_id[0]
|
||||
diplome = session_id[1]
|
||||
modalite = session_id[2]
|
||||
if year == None:
|
||||
if semobj["semestre_id"] < 0:
|
||||
rank = 1
|
||||
else:
|
||||
rank = (semobj["semestre_id"] + 1) // 2
|
||||
else:
|
||||
rank = year
|
||||
parcours = None
|
||||
groups = []
|
||||
if "groups" in etud:
|
||||
for x in etud["groups"]:
|
||||
if x["partition_name"] == "Parcours":
|
||||
parcours = x["group_name"]
|
||||
groups.append(x["group_name"])
|
||||
if parcours == None:
|
||||
parcours = ""
|
||||
parcours = get_override(semobj, "parcours", parcours)
|
||||
department = get_override(semobj, "department", department)
|
||||
rank = get_override(semobj, "rank", rank)
|
||||
diplome = get_override(semobj, "diplome", diplome)
|
||||
modalite = get_override(semobj, "modalite", modalite)
|
||||
if len(modalite) > 0 and modalite[0] == "G":
|
||||
goal = modalite.split(":")[1:]
|
||||
modalite = None
|
||||
for g in goal:
|
||||
gg = g.split("=")
|
||||
# print(f"Looking for {gg[0]} yielding {gg[1]} out of {groupes}")
|
||||
if gg[0] in groups:
|
||||
modalite = gg[1]
|
||||
nick = conf_value("nick")
|
||||
def nick_replace(
|
||||
department, diplome, rank, modalite, parcours, nick, year=Options.base_year
|
||||
):
|
||||
if len(department) > 0:
|
||||
nick = nick.replace(
|
||||
"{department}", conf_value("department_separator") + department
|
||||
@ -363,30 +353,96 @@ def analyse_student(semobj, etud, year=None):
|
||||
nick = nick.replace("{diplome}", conf_value("diplome_separator") + diplome)
|
||||
else:
|
||||
nick = nick.replace("{diplome}", "")
|
||||
if len(diplome) > 0 and diplome != "BUT":
|
||||
nick = nick.replace("{diplomenobut}", conf_value("diplome_separator") + diplome)
|
||||
else:
|
||||
nick = nick.replace("{diplomenobut}", "")
|
||||
if len(str(rank)) > 0:
|
||||
nick = nick.replace("{rank}", conf_value("rank_separator") + str(rank))
|
||||
else:
|
||||
nick = nick.replace("{rank}", "")
|
||||
nick = nick.replace(
|
||||
"{year}", conf_value("year_separator") + str(Options.base_year + rank - 1)
|
||||
)
|
||||
if diplome != "BUT":
|
||||
nick = nick.replace(
|
||||
"{yearnobut}",
|
||||
conf_value("year_separator") + str(Options.base_year + rank - 1),
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{yearnobut}", "")
|
||||
if len(modalite) > 0:
|
||||
nick = nick.replace("{modalite}", conf_value("modalite_separator") + modalite)
|
||||
else:
|
||||
nick = nick.replace("{modalite}", "")
|
||||
if len(modalite) > 0 and modalite != "FI":
|
||||
nick = nick.replace("{modaliteshort}", modalite[-1])
|
||||
else:
|
||||
nick = nick.replace("{modaliteshort}", "")
|
||||
if len(parcours) > 0:
|
||||
nick = nick.replace("{parcours}", conf_value("parcours_separator") + parcours)
|
||||
else:
|
||||
nick = nick.replace("{parcours}", "")
|
||||
formsem_department[str(semobj["id"])] = department
|
||||
if nick == " BUT 1 GEA EXT":
|
||||
print(department, diplome, rank, modalite, parcours, nick)
|
||||
sys.exit(0)
|
||||
return department, diplome, rank, modalite, parcours, nick
|
||||
|
||||
|
||||
def nick(semobj, etud):
|
||||
department, diplome, rank, modalite, parcours, nick = analyse_student(semobj, etud)
|
||||
extname = "Ecand "
|
||||
if diplome == "BUT":
|
||||
extname = "EXT"
|
||||
nick = nick.replace("{ext}", extname)
|
||||
return nick
|
||||
|
||||
|
||||
def analyse_student(semobj, etud, univ_year=None):
|
||||
"""Returns the final (department,diplome,rank,modalite,parcours,nickname,displayname) tuple from etudid in semid, taking into accounts overrides."""
|
||||
session_id = semobj["session_id"].split("-")
|
||||
year = str(semobj["annee_scolaire"])
|
||||
department = session_id[0]
|
||||
diplome = session_id[1]
|
||||
modalite = session_id[2]
|
||||
if univ_year == None:
|
||||
if semobj["semestre_id"] < 0:
|
||||
rank = 1
|
||||
else:
|
||||
rank = (semobj["semestre_id"] + 1) // 2
|
||||
else:
|
||||
rank = univ_year
|
||||
parcours = None
|
||||
groups = []
|
||||
if "groups" in etud:
|
||||
for x in etud["groups"]:
|
||||
if x["partition_name"] == "Parcours":
|
||||
parcours = x["group_name"]
|
||||
groups.append(x["group_name"])
|
||||
if parcours == None:
|
||||
parcours = ""
|
||||
parcours = get_override(semobj, "parcours", parcours)
|
||||
department = get_override(semobj, "department", department)
|
||||
rank = get_override(semobj, "rank", rank)
|
||||
diplome = get_override(semobj, "diplome", diplome)
|
||||
modalite = get_override(semobj, "modalite", modalite)
|
||||
formsem_department[str(semobj["id"])] = department
|
||||
if len(modalite) > 0 and modalite[0] == "G":
|
||||
goal = modalite.split(":")[1:]
|
||||
modalite = None
|
||||
for g in goal:
|
||||
gg = g.split("=")
|
||||
# print(f"Looking for {gg[0]} yielding {gg[1]} out of {groupes}")
|
||||
if gg[0] in groups:
|
||||
modalite = gg[1]
|
||||
nick = conf_value("nick")
|
||||
nick = nick_replace(department, diplome, rank, modalite, parcours, nick, year)
|
||||
displayname = conf_value("displayname")
|
||||
displayname = nick_replace(
|
||||
department, diplome, rank, modalite, parcours, displayname, year
|
||||
)
|
||||
return department, diplome, rank, modalite, parcours, nick, displayname
|
||||
|
||||
|
||||
def get_nick(semobj, etud):
|
||||
department, diplome, rank, modalite, parcours, nick, displayname = analyse_student(
|
||||
semobj, etud
|
||||
)
|
||||
return nick, displayname
|
||||
|
||||
|
||||
def get_dept_from_sem(semid):
|
||||
return formsem_department[str(semid)]
|
||||
|
||||
@ -448,10 +504,11 @@ def analyse_depts():
|
||||
studentsummary["modalite"] = {} # modalite
|
||||
studentsummary["parcours"] = {} # parcours
|
||||
studentsummary["nickname"] = {} # nick
|
||||
studentsummary["displayname"] = {} # display name
|
||||
studentsummary["dept"] = dept # useful when merging students
|
||||
studentsummary["bac"] = "" # usually
|
||||
department, diplome, rank, modalite, parcours, nick = analyse_student(
|
||||
sem, etud, year
|
||||
department, diplome, rank, modalite, parcours, nick, displayname = (
|
||||
analyse_student(sem, etud, year)
|
||||
)
|
||||
if "bac" in etud["admission"]:
|
||||
studentsummary["bac"] = etud["admission"]["bac"]
|
||||
@ -460,7 +517,9 @@ def analyse_depts():
|
||||
bacs.add(studentsummary["bac"])
|
||||
# We skip non-techno students if we are in techno mode
|
||||
# If we want a mixed reporting, maybe we should change this
|
||||
if Options.techno and studentsummary["bac"][:2] != "ST": # TODO: change this
|
||||
if (
|
||||
Options.techno and studentsummary["bac"][:2] != "ST"
|
||||
): # TODO: change this
|
||||
continue
|
||||
if bucket in studentsummary["cursus"]:
|
||||
semestreerreur = int(bucket) + 1
|
||||
@ -480,6 +539,7 @@ def analyse_depts():
|
||||
studentsummary["modalite"][bucket] = modalite
|
||||
studentsummary["parcours"][bucket] = parcours
|
||||
studentsummary["nickname"][bucket] = nick
|
||||
studentsummary["displayname"][bucket] = displayname
|
||||
studentsummary["debug"] = etud["sort_key"] # TODO: REMOVE
|
||||
studentsummary["unid"] = etud["code_nip"]
|
||||
cohort_nip.add(etud["code_nip"])
|
||||
@ -491,6 +551,7 @@ analyse_depts()
|
||||
|
||||
def allseeingodin():
|
||||
"""This function changes the student lists by peeking in the past and the future to know which students come from another cohort or go into a later cohort."""
|
||||
displaynames = {}
|
||||
oldstudents = {}
|
||||
oldstudentslevel = {}
|
||||
futurestudents = {}
|
||||
@ -500,6 +561,9 @@ def allseeingodin():
|
||||
for semid in oldsems:
|
||||
sem = cache["sem"][semid]
|
||||
semlevel = sem["semestre_id"]
|
||||
# For a while, some people registered former semesters (in other places) with "EXT" modalite for a fake semester
|
||||
if sem["modalite"] == "EXT": # Ignore EXT modalite
|
||||
continue
|
||||
semlevel = abs(semlevel)
|
||||
dept = oldsemsdept[semid]
|
||||
etuds = get_etuds_from_formsem(dept, semid)
|
||||
@ -509,7 +573,9 @@ def allseeingodin():
|
||||
continue
|
||||
if nip not in oldstudentslevel or semlevel > oldstudentslevel[nip]:
|
||||
oldstudentslevel[nip] = semlevel
|
||||
oldstudents[nip] = [semid, nick(sem, etud)]
|
||||
nick_t, disp_t = get_nick(sem, etud)
|
||||
oldstudents[nip] = [semid, nick_t]
|
||||
displaynames[nick_t] = disp_t
|
||||
for semid in futuresems:
|
||||
sem = cache["sem"][semid]
|
||||
if sem["formation"]["type_parcours"] != 700:
|
||||
@ -525,7 +591,7 @@ def allseeingodin():
|
||||
continue
|
||||
if nip not in futurestudentslevel or semlevel > futurestudentslevel[nip]:
|
||||
futurestudentslevel[nip] = semlevel
|
||||
futurestudents[nip] = nick(sem, etud)
|
||||
futurestudents[nip], tmp = get_nick(sem, etud)
|
||||
|
||||
unification = {}
|
||||
|
||||
@ -576,6 +642,7 @@ def allseeingodin():
|
||||
"modalite",
|
||||
"parcours",
|
||||
"nickname",
|
||||
"displayname",
|
||||
"old",
|
||||
"oldsem",
|
||||
):
|
||||
@ -584,14 +651,67 @@ def allseeingodin():
|
||||
if bucket not in base[skey]:
|
||||
base[skey][bucket] = supp[skey][bucket]
|
||||
del student[suppidx]
|
||||
foundfirst = False
|
||||
# Ensure all cursus are continuous
|
||||
for etudid in student:
|
||||
etud = student[etudid]
|
||||
foundfirst = False
|
||||
foundlast = False
|
||||
fillblanks = None
|
||||
there = -1
|
||||
for i in range(6):
|
||||
if str(i) in etud["cursus"]:
|
||||
if foundfirst and foundlast:
|
||||
fillblanks = [there, i]
|
||||
else:
|
||||
foundfirst = True
|
||||
here = i
|
||||
else:
|
||||
if not foundfirst:
|
||||
continue
|
||||
foundlast = True
|
||||
there = i
|
||||
if fillblanks is not None:
|
||||
for i in range(fillblanks[0] - 1, fillblanks[1]):
|
||||
bucket = str(i)
|
||||
if bucket not in etud["cursus"]:
|
||||
etud["etudid"][bucket] = etudid
|
||||
etud["cursus"][bucket] = -1
|
||||
etud["pseudodept"][bucket] = "OUT"
|
||||
etud["diplome"][bucket] = "OUT"
|
||||
etud["rank"][bucket] = (i // 2) + 1
|
||||
etud["modalite"][bucket] = "FI"
|
||||
etud["parcours"][bucket] = ""
|
||||
etud["nickname"][bucket] = "OUT" + str(etud["rank"][bucket])
|
||||
etud["displayname"][bucket] = "Césure"
|
||||
displaynames[etud["nickname"][bucket]] = etud["displayname"][bucket]
|
||||
return displaynames
|
||||
|
||||
|
||||
allseeingodin()
|
||||
displaynames = allseeingodin()
|
||||
|
||||
strange_cases = []
|
||||
next = {}
|
||||
nextnick = {}
|
||||
|
||||
|
||||
def prepare_display(displaynames):
|
||||
for etudid in student:
|
||||
for semlevel in range(5):
|
||||
if str(semlevel) in student[etudid]["nickname"]:
|
||||
a = student[etudid]["nickname"][str(semlevel)]
|
||||
b = student[etudid]["displayname"][str(semlevel)]
|
||||
if a in displaynames:
|
||||
if b != displaynames[a]:
|
||||
die("{a} will be displayed as {b} or {displaynames[a]} !", 6)
|
||||
else:
|
||||
displaynames[a] = b
|
||||
return displaynames
|
||||
|
||||
|
||||
displaynames = prepare_display(displaynames)
|
||||
|
||||
|
||||
for etudid in student.keys():
|
||||
etud = student[etudid]
|
||||
cursus_array = [None] * 6
|
||||
@ -712,6 +832,15 @@ unknown = {}
|
||||
entries = {}
|
||||
redirs = {}
|
||||
|
||||
finals = {
|
||||
"FAIL": "✘",
|
||||
"RED": "↩",
|
||||
"QUIT": "↴",
|
||||
"+DUT": "➡",
|
||||
"DIPLOME": "✔",
|
||||
"?": "?",
|
||||
}
|
||||
|
||||
for d in depts:
|
||||
badred[d] = 0
|
||||
goodred[d] = 0
|
||||
@ -764,46 +893,48 @@ for etudid in student.keys():
|
||||
f"REDI{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
|
||||
)
|
||||
if resultyear == None:
|
||||
finaloutput = "?" + etud["nickshort"][lastyear]
|
||||
finaloutput = "?"
|
||||
unknown[ddd] += 1
|
||||
strangecases.append(
|
||||
f"????{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
|
||||
)
|
||||
elif resultyear in ("RAT", "ATJ"):
|
||||
finaloutput = "?" + etud["nickshort"][lastyear]
|
||||
finaloutput = "?"
|
||||
unknown[ddd] += 1
|
||||
strangecases.append(
|
||||
f"ATTE{lastyear} {SCODOC_SERVER}/{ddd}/Scolarite/fiche_etud?etudid={etudid}"
|
||||
)
|
||||
elif resultyear in ("RED", "ABL", "ADSUP"):
|
||||
finaloutput = "RED " + etud["nickshort"][lastyear]
|
||||
finaloutput = "RED"
|
||||
checkred = True
|
||||
elif lastyear == 3 and resultyear in ("ADM", "ADJ"):
|
||||
finaloutput = "DIPLOME " + etud["nickshort"][lastyear]
|
||||
finaloutput = "DIPLOME"
|
||||
diploma[ddd] += 1
|
||||
elif lastyear == 2 and resultyear in ("ADM", "ADJ"):
|
||||
finaloutput = "+DUT " + etud["nickshort"][lastyear]
|
||||
finaloutput = "+DUT"
|
||||
reor2[ddd] += 1
|
||||
elif resultyear in ("PAS1NCI", "PASD"):
|
||||
finaloutput = "QUIT " + etud["nickshort"][lastyear]
|
||||
finaloutput = "QUIT"
|
||||
reor1[ddd] += 1
|
||||
elif lastyear < 2 and resultyear in ("ADM", "ADJ"):
|
||||
finaloutput = "QUIT " + etud["nickshort"][lastyear]
|
||||
finaloutput = "QUIT"
|
||||
reor1[ddd] += 1
|
||||
elif resultyear in ("NAR", "DEM", "DEF", "ABAN"):
|
||||
finaloutput = "FAIL " + etud["nickshort"][lastyear]
|
||||
finaloutput = "FAIL"
|
||||
failure[ddd] += 1
|
||||
elif resjury["annee"]["annee_scolaire"] != Options.base_year + lastyear - 1:
|
||||
finaloutput = "RED " + etud["nickshort"][lastyear]
|
||||
finaloutput = "RED"
|
||||
checkred = True
|
||||
if checkred:
|
||||
if "future" not in etud:
|
||||
# print(f"// Mauvais redoublement : {etudid}")
|
||||
badred[ddd] += 1
|
||||
finaloutput = "FAIL" + finaloutput[3:]
|
||||
finaloutput = "FAIL"
|
||||
else:
|
||||
goodred[ddd] += 1
|
||||
etud["nickshort"][lastyear + 1] = finaloutput
|
||||
output = finaloutput + " " + etud["nickshort"][lastyear]
|
||||
etud["nickshort"][lastyear + 1] = output
|
||||
displaynames[output] = finals[finaloutput] + etud["nickshort"][lastyear]
|
||||
(firstsem, firstyear) = (
|
||||
(etud["short"][1], 1)
|
||||
if etud["short"][1] != None
|
||||
@ -814,7 +945,7 @@ for etudid in student.keys():
|
||||
firstdept = cache["sem"][firstsem]["departement"]["acronym"]
|
||||
if "old" in etud:
|
||||
yearold = cache["sem"][etud["oldsem"]]["annee_scolaire"]
|
||||
etud["nickshort"][firstyear - 1] = etud["old"] + " " + str(yearold)
|
||||
etud["nickshort"][firstyear - 1] = etud["old"]
|
||||
yy = yearold
|
||||
delta = firstyear + Options.base_year - yy - 2
|
||||
for i in range(delta, firstyear - 1):
|
||||
@ -834,53 +965,13 @@ for etudid in student.keys():
|
||||
rank = etud["rank"][startsem]
|
||||
modalite = etud["modalite"][startsem]
|
||||
parcours = etud["parcours"][startsem]
|
||||
nick = conf_value("extnick")
|
||||
if len(department) > 0:
|
||||
nick = nick.replace(
|
||||
"{department}", conf_value("department_separator") + department
|
||||
nick = "EXT" + conf_value("nick")
|
||||
nick = nick_replace(department, diplome, rank, modalite, parcours, nick)
|
||||
displayname = conf_value("extnick")
|
||||
displayname = nick_replace(
|
||||
department, diplome, rank, modalite, parcours, displayname
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{department}", "")
|
||||
if len(department) > 0 and len(depts) > 1:
|
||||
nick = nick.replace(
|
||||
"{multidepartment}", conf_value("department_separator") + department
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{multidepartment}", "")
|
||||
if len(diplome) > 0:
|
||||
nick = nick.replace("{diplome}", conf_value("diplome_separator") + diplome)
|
||||
else:
|
||||
nick = nick.replace("{diplome}", "")
|
||||
if len(diplome) > 0 and diplome != "BUT":
|
||||
nick = nick.replace(
|
||||
"{diplomenobut}", conf_value("diplome_separator") + diplome
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{diplomenobut}", "")
|
||||
if len(str(rank)) > 0:
|
||||
nick = nick.replace("{rank}", conf_value("rank_separator") + str(rank))
|
||||
else:
|
||||
nick = nick.replace("{rank}", "")
|
||||
if len(modalite) > 0:
|
||||
nick = nick.replace(
|
||||
"{modalite}", conf_value("modalite_separator") + modalite
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{modalite}", "")
|
||||
if len(modalite) > 0 and modalite != "FI":
|
||||
nick = nick.replace("{modaliteshort}", modalite[-1])
|
||||
else:
|
||||
nick = nick.replace("{modaliteshort}", "")
|
||||
if len(parcours) > 0:
|
||||
nick = nick.replace(
|
||||
"{parcours}", conf_value("parcours_separator") + parcours
|
||||
)
|
||||
else:
|
||||
nick = nick.replace("{parcours}", "")
|
||||
if diplome != "BUT":
|
||||
nick = "Ecand " + nick
|
||||
else:
|
||||
nick = "EXT" + nick
|
||||
displaynames[nick] = displayname
|
||||
etud["nickshort"][firstyear - 1] = nick
|
||||
for i in range(0, firstyear - 1):
|
||||
etud["nickshort"][i] = nick + "*" * (firstyear - 1 - i)
|
||||
@ -902,93 +993,6 @@ for etudid in student.keys():
|
||||
bags[i][nstart][nend] += 1
|
||||
|
||||
|
||||
# layers = [[], [], [], [], []]
|
||||
# alllayers = []
|
||||
# flatbags = []
|
||||
# for i in range(4):
|
||||
# for u in bags[i]:
|
||||
# if u not in alllayers:
|
||||
# alllayers.append(u)
|
||||
# layers[i].append(u)
|
||||
# for v in bags[i][u]:
|
||||
# if v not in alllayers:
|
||||
# alllayers.append(v)
|
||||
# layers[i + 1].append(v)
|
||||
# flatbags.append([u, v, bags[i][u][v]])
|
||||
# allowed = []
|
||||
# nextallowed = [[], [], [], [], []]
|
||||
# weights = {}
|
||||
|
||||
|
||||
# orders = conf_value("orders")
|
||||
|
||||
# x = set(alllayers)
|
||||
# y = set()
|
||||
# for i in orders:
|
||||
# y = y.union(set(i))
|
||||
|
||||
# for i in range(5):
|
||||
# if len(orders[i]) > 0:
|
||||
# allowed.append(orders[i][0])
|
||||
# for j in orders[i]:
|
||||
# if j in alllayers:
|
||||
# nextallowed[i].append(j)
|
||||
# for j, k in enumerate(orders[i]):
|
||||
# weights[k] = j + 1
|
||||
# for u in layers[i]:
|
||||
# if u not in allowed and u not in nextallowed[i]:
|
||||
# allowed.append(u)
|
||||
# else:
|
||||
# for i in range(5):
|
||||
# allowed.extend(layers[i])
|
||||
|
||||
|
||||
# for bag in flatbags:
|
||||
# w = 0
|
||||
# if bag[0] in weights:
|
||||
# w += weights[bag[0]]
|
||||
# if bag[1] in weights:
|
||||
# w += weights[bag[1]]
|
||||
# bag.append(w)
|
||||
# flatbags = sorted(flatbags, key=lambda x: x[-1])
|
||||
|
||||
|
||||
# orderedflatbags = []
|
||||
# finallayers = [[], [], [], [], []]
|
||||
|
||||
# while len(flatbags) > 0:
|
||||
# gotone = False
|
||||
# for x in flatbags:
|
||||
# if x[0] in allowed and x[1] in allowed:
|
||||
# # print(f"{x} est pris")
|
||||
# gotone = True
|
||||
# orderedflatbags.append(x)
|
||||
# flatbags.remove(x)
|
||||
# # print(f"Choosing {x}")
|
||||
# for i in range(5):
|
||||
# if x[0] in layers[i] and x[0] not in finallayers[i]:
|
||||
# finallayers[i].append(x[0])
|
||||
# if i < 4 and x[1] in layers[i + 1] and x[1] not in finallayers[i + 1]:
|
||||
# finallayers[i + 1].append(x[1])
|
||||
# if x[0] in nextallowed[i]:
|
||||
# # print(f"[{i}] Removing {x[0]} from {nextallowed[i]}")
|
||||
# nextallowed[i].remove(x[0])
|
||||
# if x[1] in nextallowed[i]:
|
||||
# # print(f"[{i}] Removing {x[1]} from {nextallowed[i]}")
|
||||
# nextallowed[i].remove(x[1])
|
||||
# # print(f"[{i}] {nextallowed[i]}")
|
||||
# if len(nextallowed[i]) > 0 and nextallowed[i][0] not in allowed:
|
||||
# # print(f"[{i}] Allowing now {nextallowed[i][0]}")
|
||||
# allowed.append(nextallowed[i][0])
|
||||
# break
|
||||
# if not gotone:
|
||||
# print("BUG")
|
||||
# print(flatbags)
|
||||
# print("---", allowed)
|
||||
# print(nextallowed)
|
||||
# sys.exit(3)
|
||||
|
||||
|
||||
def node_color(x):
|
||||
color = colors["NORMAL"]
|
||||
if x[0:4] == "FAIL":
|
||||
@ -1012,78 +1016,6 @@ def node_color(x):
|
||||
return color
|
||||
|
||||
|
||||
# def printout():
|
||||
# with open(f"sankeymatic_{orderkey}.txt", "w") as fout:
|
||||
|
||||
# def output(*a, **b):
|
||||
# b["file"] = fout
|
||||
# print(*a, **b)
|
||||
|
||||
# date_actuelle = datetime.now()
|
||||
# date_formatee = date_actuelle.strftime("%m/%d/%Y %H:%M:%S")
|
||||
|
||||
# output(
|
||||
# f"// SankeyMATIC diagram inputs - Saved: {date_formatee}\n// https://sankeymatic.com/build/\n\n// === Nodes and Flows ===\n\n"
|
||||
# )
|
||||
|
||||
# output("// THEME INFO")
|
||||
# for c, cc in colors.items():
|
||||
# output(f"// !{c}:{cc}")
|
||||
# output()
|
||||
|
||||
# allnodes = []
|
||||
# for y in orderedflatbags:
|
||||
# output(f"{y[0]} [{y[2]}] {y[1]}")
|
||||
# allnodes.append(y[0])
|
||||
# allnodes.append(y[1])
|
||||
# allnodes = list(set(allnodes))
|
||||
|
||||
# nodes = {}
|
||||
# for x in allnodes:
|
||||
# color = node_color(x)
|
||||
# if len(color):
|
||||
# nodes[x] = color
|
||||
|
||||
# for u in sorted(nodes.keys()):
|
||||
# output(f":{u} {nodes[u]}")
|
||||
|
||||
# height = conf_value("height")
|
||||
# width = conf_value("width")
|
||||
# output("\n\n// === Settings ===\n")
|
||||
# output(f"size w {width}")
|
||||
# output(f" h {height}")
|
||||
# with open("trailer.txt", "r") as fichier:
|
||||
# contenu = fichier.read()
|
||||
# output(contenu)
|
||||
# for ddd in depts:
|
||||
# if entries[ddd] == 0:
|
||||
# continue
|
||||
# p1 = round(100 * diploma[ddd] / entries[ddd])
|
||||
# p2 = round(100 * (diploma[ddd] + reor2[ddd]) / entries[ddd])
|
||||
# p3 = round(100 * (failure[ddd] / entries[ddd]))
|
||||
# p4 = round(100 * (failure[ddd] + badred[ddd] + reor1[ddd]) / entries[ddd])
|
||||
|
||||
# output(f"// Département {ddd}")
|
||||
# output(f"// {entries[ddd]} Entrées")
|
||||
# output(f"// {diploma[ddd]} Diplômes")
|
||||
# output(f"// {reor2[ddd]} DUT")
|
||||
# output(f"// {p1}-{p2}% de réussite")
|
||||
# output(f"// {goodred[ddd]} Redoublements")
|
||||
# output(f"// {reor1[ddd]} départs de la formation")
|
||||
# output(f"// {badred[ddd]} redoublements autorisés non actés")
|
||||
# output(f"// {failure[ddd]} échecs")
|
||||
# output(f"// {p3}-{p4}% d'échecs")
|
||||
# output(f"// {unknown[ddd]} inconnus")
|
||||
# for x in strangecases:
|
||||
# output(f"// {x}")
|
||||
|
||||
# output(f'// orders["{orderkey}"] = {finallayers}')
|
||||
# output(f"// bacs: {bacs}")
|
||||
|
||||
|
||||
# printout()
|
||||
|
||||
|
||||
def textwidth(text, font="Arial", fontsize=14):
|
||||
try:
|
||||
import cairo
|
||||
@ -1112,8 +1044,6 @@ def crossweight(node_position, node_layer, edges):
|
||||
return w
|
||||
|
||||
|
||||
|
||||
|
||||
def genetic_optimize(node_position, node_layer, edges):
|
||||
oldcandidates = []
|
||||
l_indices = list(range(5))
|
||||
@ -1306,7 +1236,20 @@ def printsvg():
|
||||
k["out"] = lho
|
||||
if lhi != lho and lhi * lho != 0:
|
||||
print(f"BUG1: {j} {k} {lhi} {lho}")
|
||||
print(json.dumps(layer_structure, indent=2))
|
||||
print(json.dumps(node_structure, indent=2))
|
||||
ls["inout"] += k["size"]
|
||||
if height == 0:
|
||||
minheight = 0
|
||||
for i in range(5):
|
||||
ls = layer_structure[i]
|
||||
new_minheight = spacing * (ls["num"] - 1) + 2 * hmargin + 2 * ls["inout"]
|
||||
if new_minheight > minheight:
|
||||
minheight = new_minheight
|
||||
height = (1 + (minheight // 150)) * 150
|
||||
|
||||
for i in range(5):
|
||||
ls = layer_structure[i]
|
||||
ls["density"] = ls["inout"] / (
|
||||
spacing + height - spacing * ls["num"] - 2 * hmargin
|
||||
)
|
||||
@ -1353,7 +1296,7 @@ def printsvg():
|
||||
stroke="black",
|
||||
)
|
||||
g1.append(r)
|
||||
nw = textwidth(n, "Arial", fontsize_name) * unit_ratio
|
||||
nw = textwidth(displaynames[n], "Arial", fontsize_name) * unit_ratio
|
||||
cw = textwidth(str(ns["size"]), "Arial", fontsize_count) * unit_ratio
|
||||
gw = nw + cw + padding
|
||||
ggw = gw + 2 * padding
|
||||
@ -1370,7 +1313,7 @@ def printsvg():
|
||||
cxpos += gw / 2 + padding + thickness
|
||||
rxpos += gw / 2 + padding + thickness
|
||||
t = drawsvg.Text(
|
||||
n,
|
||||
displaynames[n],
|
||||
str(fontsize_name) + "pt",
|
||||
nxpos,
|
||||
ypos + fontsize_name / 2,
|
||||
|
@ -2,7 +2,7 @@ certifi==2024.8.30
|
||||
charset-normalizer==3.4.0
|
||||
drawsvg==2.4.0
|
||||
idna==3.10
|
||||
pycairo==1.27.0
|
||||
pycairo==1.27.0;platform_system != "Windows"
|
||||
python-dotenv==1.0.1
|
||||
requests==2.32.3
|
||||
urllib3==2.2.3
|
||||
|
Loading…
Reference in New Issue
Block a user