Sinus example

[Ax3man]

Hi guys,

I'm very interested in running RNN in R, and I think the simplicity of the interface in this package is very appealing. However, I can't seem to get one of your examples working, neither with the CRAN version or the GitHub version.

In the sinus example, the model just predicts an almost flat line at the mean (something that I observed when running a model on my own data as well). I thought the prediction should follow Y to some extent, and so the black and red lines should be somewhat similar.

Am I misunderstanding the purpose of the example?

To clarify, unless I'm missing something, this is a report of a broken example.

[faltinl]

Hi,-
the program ist very sensible w.r.t. seed and numepochs. Using the following setup it produced the picture shown subsequently. Here, you will see more or less what you expected, the 2 rhs boxes showing the prediction.
According to my trials there are seeds which will not lead to convergence; others may produce just a slightly wiggly red line. You will notice from the picture below that even after 500 iterations, the rnn is far from having reached a steady state.

---

library("rnn")
library("sigmoid")

synthetic time serie prediction

set.seed(3)
sample_dim <- 9
time_dim <- 200
X <- data.frame()
Y <- data.frame()

Genereate a bias in phase

bias_phase <- rnorm(sample_dim)

Generate a bias in frequency

bias_frequency = runif(sample_dim,min=5,max=25)

Generate the noisy time series, cosinus for X and sinus for Y,

with a random bias in phase and in frequency

for(i in seq(sample_dim)){
X <- rbind(X,sin(seq(time_dim)/bias_frequency[i]+bias_phase[i])+rnorm(time_dim,mean=0,sd=0.2))
Y <- rbind(Y,cos(seq(time_dim)/bias_frequency[i]+bias_phase[i])+rnorm(time_dim,mean=0,sd=0.2))
}
X <- as.matrix(X)
Y <- as.matrix(Y)

Normalize between 0 and 1 for the sigmoid

X <- (X-min(X))/(max(X)-min(X))
Y <- (Y-min(Y))/(max(Y)-min(Y))

Train the model with all but the 2 last samples

note: network_type = "rnn" is default

model <- trainr(Y = Y[seq(sample_dim-2),],X = X[seq(sample_dim-2),],
learningrate = 0.05, hidden_dim =c(16),
network_type = "rnn", numepochs=500, use_bias = F,
batch_size = 1, momentum = 0, learningrate_decay = 1)

# Plot using testing data and predict all samples

layout(cbind(seq(sample_dim-2),c((sample_dim-1):sample_dim,rep(sample_dim+1,sample_dim-4))))
par(mar=c(1.5,2,1,1),xaxt="s",yaxt="s",mgp=c(1.5,0.5,0),oma=c(0,0,4,0))
for(i in seq(sample_dim)){
plot(X[i,],type="l",col="green",ylim=c(0,1),xlab="",ylab="")
par(new=T)
plot(Y[i,],type="l",ylim=c(0,1),xlab="",ylab="")
par(new=T)
plot(predictr(model,X)[i,],type="l",ylim=c(0,1),col="red",xlab="",ylab="")
}
plot(colMeans(model$error),type="l",xlab="",ylab="",xlim=c(1,500))
title(main="Left: Training time series - Right: Test time series and learning curve
Green: X, noisy cosinus - Black: Y, noisy sinus - Red: network prediction
The network learns to represent the bias in phases and frequencies",outer=T)

---

---

I am using R3.3.4 under RStudio Version 1.1.383 and WIN10.
Cheers,
Leo