| Title: | Visualisation And Analysis During a Survey Field |
|---|---|
| Description: | vizsurvey is an R package designed to streamline the quality assessment of survey data by providing intuitive visual diagnostics through an interactive dashboard. vizsurvey is especially useful for institutions or researchers conducting large-scale surveys with multiple interviewers, enabling a fast and systematic overview of data quality over time. |
| Authors: | Thomas Delclite [aut, cre], Adrien Mierop [aut] |
| Maintainer: | Thomas Delclite <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.4.5 |
| Built: | 2026-10-28 13:54:58 UTC |
| Source: | https://github.com/tdelc/vizsurvey |
Build the df_timer of interviewer
build_df_timer_intvwr(df_timer_intv, cfg)build_df_timer_intvwr(df_timer_intv, cfg)
df_timer_intv |
df_timer_intv |
cfg |
config list |
data.frame
Classify all variable of a data.frame
classify_df(df, threhold = 15)classify_df(df, threhold = 15)
df |
A data frame |
threhold |
Maximum number of modalities to classify variable as modal |
a data frame
classify_df(iris)classify_df(iris)
Used when var_wave (or var_filter) contains more than one variable in the configuration file, for example the year and the quarter. The complete key of a row is then "2024 / T1".
combine_vars(df, vars, sep = " / ")combine_vars(df, vars, sep = " / ")
df |
data.frame |
vars |
vector of the variable of each level |
sep |
separator between the levels |
vector of the complete keys
head(combine_vars(mtcars,c("cyl","gear")))head(combine_vars(mtcars,c("cyl","gear")))
corrections of each df of a list
correct_list_df(list_df)correct_list_df(list_df)
list_df |
list of df |
list
Create a template of configuration file
create_config( folder_path, file_name = "config.txt", name_survey = NULL, vars_discretes = NULL, vars_continuous = NULL, vars_ignore = NULL, prefix_discretes = NULL, prefix_continuous = NULL, prefix_ignore = NULL, var_wave = NULL, var_filter = NULL, var_intvwr = NULL, var_intv = NULL, var_date = NULL, var_info_geo = NULL, var_timer = NULL, var_itm_duration = NULL, var_session = NULL, duration_min_during = NULL, duration_min_inter = NULL, night_start = NULL, night_end = NULL )create_config( folder_path, file_name = "config.txt", name_survey = NULL, vars_discretes = NULL, vars_continuous = NULL, vars_ignore = NULL, prefix_discretes = NULL, prefix_continuous = NULL, prefix_ignore = NULL, var_wave = NULL, var_filter = NULL, var_intvwr = NULL, var_intv = NULL, var_date = NULL, var_info_geo = NULL, var_timer = NULL, var_itm_duration = NULL, var_session = NULL, duration_min_during = NULL, duration_min_inter = NULL, night_start = NULL, night_end = NULL )
folder_path |
folder where create the file |
file_name |
Name of the config file (config.txt by default) |
name_survey |
Name of the survey (not used) |
vars_discretes |
(optional) preset discretes variables name (VAR1,VAR2,...) |
vars_continuous |
(optional) preset continuous variables name (VAR1,VAR2,...) |
prefix_discretes |
(optional) preset prefix for discretes variables name |
prefix_continuous |
(optional) preset prefix for continuous variables name |
var_wave |
(optional) variable name of wave. Two variables can be given (c("YEAR","QUARTER")) : the second one is then a second level of wave |
var_filter |
(optional) variable name of filter. Two variables can be given : the second one is then a second level of filter |
var_intvwr |
(optional) variable name of interviewer id |
var_intv |
(optional) variable name of interview id |
var_date |
(optional) variable name of the date of the interview |
var_timer |
(optional, audit trail) variable name of the beginning of a item |
var_itm_duration |
(optional, audit trail) variable name of the duration of a item |
var_session |
(optional, audit trail) variable name of the session of a item |
duration_min_during |
(optional, audit trail) threshold of a interview duration |
duration_min_inter |
(optional, audit trail) threshold of the duration between two interviews |
night_start |
(optional, audit trail) hour of beginning of the night |
night_end |
(optional, audit trail) hour of end of the night |
create_config(".") # creation of config.txt in working directorycreate_config(".") # creation of config.txt in working directory
Create statistics from database
create_df_stats( df_, configs_, var_calculs, mod_filter = NULL, na.rm = FALSE, force_cat = FALSE )create_df_stats( df_, configs_, var_calculs, mod_filter = NULL, na.rm = FALSE, force_cat = FALSE )
df_ |
database |
var_calculs |
variable to create stats |
mod_filter |
(optional) modality to filter data |
na.rm |
include or not missing values as modality |
configs |
configs |
df data.frame
Simulate EU-SILC dataset with injected errors
create_eusilc_sim()create_eusilc_sim()
data.frame from laeken::eusilc with ids and errors
# create_eusilc_sim()# create_eusilc_sim()
Use PUF files for SILC example
create_fake_silc( vec_country = c("BE", "RO"), path_out = "inst/shiny/complete/data/SILC/" )create_fake_silc( vec_country = c("BE", "RO"), path_out = "inst/shiny/complete/data/SILC/" )
vec_country |
Vector of country to import |
path_out |
Path to export csv |
nothing, but creation of csv datasets
# create_fake_silc()# create_fake_silc()
Cut a continuous variable in n categories.
cut_safe(x, n_breaks = 5)cut_safe(x, n_breaks = 5)
x |
vector to cut |
n_breaks |
number of breaks |
vector
Remplace summarytools::dfSummary. Renvoie un data.frame prêt à afficher : une ligne par modalité (variables catégorielles) ou par statistique (variables continues), une colonne par groupe.
describe_variable( x, by = NULL, type = c("auto", "categorical", "continuous"), max_levels = 20, digits = 1 )describe_variable( x, by = NULL, type = c("auto", "categorical", "continuous"), max_levels = 20, digits = 1 )
x |
vecteur à décrire |
by |
(optionnel) vecteur de même longueur, un bloc de colonnes par valeur |
type |
"auto" (numérique et plus de 15 valeurs distinctes -> continue), "categorical" ou "continuous" |
max_levels |
nombre maximum de modalités affichées, le reste est regroupé dans "(other)" |
digits |
nombre de décimales |
Les pourcentages des modalités sont calculés sur les valeurs valides, celui de la ligne "(missing)" sur l'ensemble des lignes.
data.frame
Description d'une variable rendue en DT
describe_variable_dt(x, by = NULL, ..., caption = NULL)describe_variable_dt(x, by = NULL, ..., caption = NULL)
x |
vecteur à décrire |
by |
(optionnel) vecteur de même longueur, un bloc de colonnes par valeur |
caption |
(optionnel) titre du tableau |
datatable
Replace empty by na
empty_as_na(vec)empty_as_na(vec)
vec |
Vector of values |
Vector
airquality[which(is.na(airquality$Ozone)), "Ozone"] <- "" empty_as_na(airquality$Ozone)airquality[which(is.na(airquality$Ozone)), "Ozone"] <- "" empty_as_na(airquality$Ozone)
extract a config from key (config from load_config)
extract_config(config, key_)extract_config(config, key_)
config |
df of configuration |
key_ |
key to extract |
string
Extract timer rds files from folder
find_timer_rds(folder)find_timer_rds(folder)
folder |
folder where check the file |
path of files
tranform data from folder to config and df
folder_to_df(folder, file_pattern = "*.csv", file_config = "config.txt")folder_to_df(folder, file_pattern = "*.csv", file_config = "config.txt")
folder |
folder of databases |
file_pattern |
pattern of the databases (*.csv by default) |
file_config |
name of the configuration file (config.txt by default) |
list(df,configs)
## Not run: folder_to_df("ESS10") ## End(Not run)## Not run: folder_to_df("ESS10") ## End(Not run)
Vecteur de groupe "VAR = valeur" / "VAR != valeur" pour les comparaisons
group_vs_others(x, value, label)group_vs_others(x, value, label)
x |
Vecteur a regrouper |
value |
valeur discrétionnaire |
label |
label à utiliser |
vector
Create a heatmap
heatmap_group(df_stats, threshold = 5, color = "red2")heatmap_group(df_stats, threshold = 5, color = "red2")
df_stats |
data frame from prepa_stats function |
threshold |
threshold to show difference |
color |
color of the cells |
heatmap (ggplot)
library(laeken) data(eusilc) df_stats <- prepa_stats(eusilc, "db040") heatmap_group(df_stats, 5)library(laeken) data(eusilc) df_stats <- prepa_stats(eusilc, "db040") heatmap_group(df_stats, 5)
index_homogeneity
index_homogeneity(df, var_itw, threahold = 30)index_homogeneity(df, var_itw, threahold = 30)
df |
dataset |
var_itw |
groupe variable |
threahold |
threahold |
dataset
Check if value is integer64
is.integer64(x)is.integer64(x)
x |
Value |
Boolean
is.integer64(c(1:100)) # FALSEis.integer64(c(1:100)) # FALSE
First level of a multi-level key
key_level1(x, sep = " / ")key_level1(x, sep = " / ")
x |
vector of keys |
sep |
separator between the levels |
vector of the first level values
key_level1(c("2024 / T1","2024 / T2","2023 / T4"))key_level1(c("2024 / T1","2024 / T2","2023 / T4"))
Other levels of a multi-level key (used as label in the interface)
key_level2(x, sep = " / ")key_level2(x, sep = " / ")
x |
vector of keys |
sep |
separator between the levels |
vector of the values after the first level
key_level2(c("2024 / T1","2024 / T2","2023 / T4"))key_level2(c("2024 / T1","2024 / T2","2023 / T4"))
Keys (modalities) of a wave or filter variable
keys_vars(df, vars, var_key, level = "all", sep = " / ")keys_vars(df, vars, var_key, level = "all", sep = " / ")
df |
data.frame |
vars |
variables of each level (see vars_levels) |
var_key |
name of the variable containing the complete key |
level |
1 (first level only), 2 (complete key) or "all" (both) |
sep |
separator between the levels |
vector of keys
keys_vars(mtcars,"cyl","cyl")keys_vars(mtcars,"cyl","cyl")
List distribution of discrete variables
list_dist(df, vars_vd)list_dist(df, vars_vd)
df |
data.frame |
vars_vd |
vector of discrete variables |
list
list_dist(mtcars,c("cyl","vs","gear"))list_dist(mtcars,c("cyl","vs","gear"))
List of proportions for categorical variables
list_dist_dt(df, vars_discretes, na.rm = FALSE)list_dist_dt(df, vars_discretes, na.rm = FALSE)
df |
data.frame |
vars_discretes |
vector of categorical variable to calculate proportion |
na.rm |
Remove missing values or not ? |
list
load a config file for prepare data
load_config(file_path)load_config(file_path)
file_path |
path of the configuration file |
df
Loop of stats creation by filter
loop_stats(df, configs, var_calculs, na.rm = FALSE, force_cat = FALSE)loop_stats(df, configs, var_calculs, na.rm = FALSE, force_cat = FALSE)
df |
database |
configs |
configs |
var_calculs |
variable to create stats |
df
A row matches a key when its complete key is one of the keys, or when its first level is one of the keys (selection of a year without the quarter).
match_keys(df, vars, var_key, keys, sep = " / ")match_keys(df, vars, var_key, keys, sep = " / ")
df |
data.frame |
vars |
variables of each level (see vars_levels) |
var_key |
name of the variable containing the complete key |
keys |
vector of keys to keep |
sep |
separator between the levels |
logical vector
match_keys(mtcars,"cyl","cyl",4)match_keys(mtcars,"cyl","cyl",4)
Specific chisq test to NA and Other modality
my_chisq_test(x, varname, ldist, useNA = "ifany")my_chisq_test(x, varname, ldist, useNA = "ifany")
x |
value to procede chisq test |
varname |
name of the variable |
ldist |
named list of expected probability |
useNA |
useNA parameter for the table function |
chisq value
ldist <- list_dist(mtcars,c("cyl","gear")) sub_mtcars <- subset(mtcars,vs == 1) my_chisq_test(sub_mtcars$cyl,"cyl",ldist)ldist <- list_dist(mtcars,c("cyl","gear")) sub_mtcars <- subset(mtcars,vs == 1) my_chisq_test(sub_mtcars$cyl,"cyl",ldist)
Create a summarise of all the difference
prepa_stats(df, var_group, vars_vd = NULL, vars_vc = NULL)prepa_stats(df, var_group, vars_vd = NULL, vars_vc = NULL)
df |
data frame for the summary |
var_group |
Name of group variable |
vars_vd |
(optional) Vector of discrete variables |
vars_vc |
(optional) Vector of continuous variables |
data frame
library(laeken) data(eusilc) info_vars <- classify_df(eusilc) vars_vd <- info_vars[info_vars$type == "Modal", ]$variable vars_vc <- info_vars[info_vars$type == "Continuous", ]$variable prepa_stats(eusilc, "db040", vars_vd, vars_vc)library(laeken) data(eusilc) info_vars <- classify_df(eusilc) vars_vd <- info_vars[info_vars$type == "Modal", ]$variable vars_vc <- info_vars[info_vars$type == "Continuous", ]$variable prepa_stats(eusilc, "db040", vars_vd, vars_vc)
prepa_stats_dt
prepa_stats_dt(df, var_group, configs, na.rm = FALSE)prepa_stats_dt(df, var_group, configs, na.rm = FALSE)
df |
|
var_group |
|
configs |
|
na.rm |
data.frame
Preparation of a survey
prepa_survey( folder_path, file_pattern = "*.csv", file_config = "config.txt", na.rm = FALSE )prepa_survey( folder_path, file_pattern = "*.csv", file_config = "config.txt", na.rm = FALSE )
folder_path |
folder of survey |
file_pattern |
pattern of the databases (*.csv by default) |
file_config |
name of the configuration file (config.txt by default) |
NULL (creation of rds)
## Not run: prepa_survey("shiny/complete/ESS10") ## End(Not run)## Not run: prepa_survey("shiny/complete/ESS10") ## End(Not run)
Preparation of all surveys from a folder
prepa_surveys( folder_path, depth_folder = 1, file_pattern = "*.csv", file_config = "config.txt" )prepa_surveys( folder_path, depth_folder = 1, file_pattern = "*.csv", file_config = "config.txt" )
folder_path |
folder of the folders of survey |
depth_folder |
level of depth for the tree structure |
file_pattern |
pattern of the databases (*.csv by default) |
file_config |
name of the configuration file (config.txt by default) |
NULL (creation of rds)
## Not run: prepa_surveys("inst/extdata/SILC/HFILE") ## End(Not run)## Not run: prepa_surveys("inst/extdata/SILC/HFILE") ## End(Not run)
Shiny vizsurvey
runVizsurvey()runVizsurvey()
shinyapp
## Not run: runVizsurvey()## Not run: runVizsurvey()
Shiny vizsurvey with already prepared data
runVizsurvey_from_folder( path, path_dict = NULL, path_nomen = NULL, path_archive = NULL, data_rds_pattern = "global", depth_folder = 1 )runVizsurvey_from_folder( path, path_dict = NULL, path_nomen = NULL, path_archive = NULL, data_rds_pattern = "global", depth_folder = 1 )
data_rds_pattern |
name of the rds file contains all the data |
depth_folder |
level of depth for the tree structure |
link |
link to directory of data |
shinyapp
# We assume that config.txt, and prepa_surveys are already done here. ## Not run: runVizsurvey_from_folder("inst/extdata",depth_folder = 3)# We assume that config.txt, and prepa_surveys are already done here. ## Not run: runVizsurvey_from_folder("inst/extdata",depth_folder = 3)
Shiny vizsurvey from a R data.frame
runVizsurvey_from_r( df, vars_discretes = NULL, vars_continuous = NULL, var_wave = NULL, var_filter = NULL, var_intvwr = NULL )runVizsurvey_from_r( df, vars_discretes = NULL, vars_continuous = NULL, var_wave = NULL, var_filter = NULL, var_intvwr = NULL )
df |
data.frame |
vars_discretes |
(optional) preset of discretes variables |
vars_continuous |
(optional) preset of continuous variables |
var_wave |
(optional) name of wave variable. Two variables can be given (c("YEAR","QUARTER")) : the second one is then a second level of wave |
var_filter |
(optional) name of filter variable. Two variables can be given : the second one is then a second level of filter |
var_intvwr |
(optional) name of interviewer variable |
shinyapp
library(laeken) data(eusilc) set.seed(123) eusilc$NR_ITW <- paste(eusilc$db040,sample(1:5,nrow(eusilc),replace = TRUE),sep="-") ## Not run: runVizsurvey_from_r(eusilc,var_intvwr = "NR_ITW",var_filter = "db040")library(laeken) data(eusilc) set.seed(123) eusilc$NR_ITW <- paste(eusilc$db040,sample(1:5,nrow(eusilc),replace = TRUE),sep="-") ## Not run: runVizsurvey_from_r(eusilc,var_intvwr = "NR_ITW",var_filter = "db040")
Robust Scale of a varible with IQR
scale_IQR(x)scale_IQR(x)
x |
vector |
vector
head(scale_IQR(iris$Sepal.Length))head(scale_IQR(iris$Sepal.Length))
calculate isoforest score from df
score_isoforest(df)score_isoforest(df)
df |
database |
vector
score_isoforest(iris[sapply(iris, is.numeric)])score_isoforest(iris[sapply(iris, is.numeric)])
Kept compatible with the global.rds prepared before the multi-level option : vars_wave (or vars_filter) is then missing and var_wave is used.
vars_levels(configs, type = "wave")vars_levels(configs, type = "wave")
configs |
list of configuration |
type |
"wave" or "filter" |
vector of the variables of each level
vars_levels(list(var_wave = "YEAR"),"wave")vars_levels(list(var_wave = "YEAR"),"wave")