BRAIL (Block Randomized Adaptive Iterative Lasso) is the key algorithm in MixedGraphs package.

Algorithm

Repeat through the blocks. For each block of the data, BRAIL will apply the adaptive lasso method to many bootstrap samples of the data. The levels of adaptive regularization is defined as, \begin{align*} \lambda_{k,j}^{(t)} = \begin{cases} \eta^{(t)}_k & \text{if } \hat{\beta}^{(t-1)}_{k,j} \neq 0, \\ 2\eta^{(t)}_k & \text{otherwise} \end{cases} \end{align*} where \begin{equation*} \eta^{(t)}_k=\frac{\Lambda_{\max}(\widehat{\mathbf{\Theta}}^{(t-1)}_k)}{\Lambda_{\max}(\mathbf{X}^T\mathbf{X})} \ \frac{1}{\sqrt{n} } \|\hat{\mathbf{\beta}}^{(t-1)}_{k}\|_{2} \sqrt{\frac{\log( p_{k})}{n}\hspace{3pt} \|\hat{\mathbf{\beta}}^{(t-1)}_{k} \|_{0} } \end{equation*}

The weight of the adaptive lasso is \(\gamma_j\lambda^{(t)}_{k,j}\), where \(\gamma_j \buildrel{iid}\over\sim \mathcal{U}([0.5, 1.5])\). BRAIL will select out the variable which support ratio reach the user input \(\tau\). The selected variables will do the Ridge regression to get the more accurate coeffients.

We will end the loop until the coeffients for all blocks stop changing.

Parallelization

We use foreach package to implement the parallelization of BRAIL and Mixedgraph, which supports different backend, Multicore, Cluster, Cloud based (via Redis). User can register the backend according to theis specific needs and system. Many packages are available like doParallel, doMC, doSNOW, and etc. Each parallel backend has its own registration function. For example, when register doParalle, we could

library(doParallel)
registerDoParallel(cores=2)

and for doMC,

library(doMC)
registerDoMC(2)

for doSNOW,

library(doSNOW)
cl <- makeCluster(2, type="SOCK")
registerDoSNOW(cl)

Code example:

library("MixedGraphs")
n <- 100
p <- 50
k <- 2
set.seed(2)
beta <- numeric(p * k)
beta[1:8] <- 5
beta[51:53] <- 3
X <- lapply(1 : k, function(x) {
    matrix(rnorm(n * p), n, p)
})
y <- do.call(cbind, X) %*% beta + rnorm(n)
brail_test <- BRAIL(X, y, family = "G")

## Warning: executing %dopar% sequentially: no parallel backend registered

## [1] "Truly coefficients: "

##   [1] 5 5 5 5 5 5 5 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [36] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [71] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

## [1] "BRAIL coefficients: "

## $coefficients
## $coefficients[[1]]
##  [1] 4.963742 5.212161 5.086796 4.896095 5.216079 4.751499 4.960975
##  [8] 5.018468 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [15] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [22] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [29] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [36] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [43] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [50] 0.000000
## 
## $coefficients[[2]]
##  [1] 3.059511 2.955202 2.961144 0.000000 0.000000 0.000000 0.000000
##  [8] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [15] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [22] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [29] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [36] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [43] 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
## [50] 0.000000
## 
## 
## $scores
## $scores[[1]]
##  [1] 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000 0.000 0.000
## [12] 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.005 0.000
## [23] 0.000 0.000 0.000 0.000 0.000 0.005 0.000 0.000 0.005 0.000 0.000
## [34] 0.000 0.000 0.000 0.000 0.000 0.000 0.005 0.000 0.000 0.000 0.005
## [45] 0.000 0.000 0.000 0.025 0.000 0.005
## 
## $scores[[2]]
##  [1] 1.000 1.000 1.000 0.020 0.025 0.115 0.000 0.005 0.010 0.010 0.050
## [12] 0.020 0.005 0.005 0.050 0.090 0.005 0.025 0.080 0.015 0.010 0.000
## [23] 0.000 0.005 0.100 0.005 0.020 0.000 0.000 0.005 0.010 0.055 0.040
## [34] 0.025 0.010 0.035 0.005 0.055 0.055 0.015 0.065 0.030 0.015 0.040
## [45] 0.000 0.030 0.030 0.065 0.005 0.020