Guobao Li

2018-05-06, 10:24

Matthias Boehm

2018-05-06, 20:09

Guobao Li

2018-05-07, 18:26

Matthias Boehm

2018-05-10, 19:47

Guobao Li

2018-05-10, 21:44

- SystemML
- mail # dev
- Questions about MNIST LeNet example

Hi Matthias,

I'm currently reading the dml script MNIST LeNet example and got some

questions. I hope that you could help me out of them.

1) Is it possible to define a matrix containing the variables? Because I'm

wondering how to represent the model as a parameter for the "paramserv"

function.

2) What is the role of bias? Why we need it?

Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

to get some feedback. Thanks!

Regards,

Guobao

I'm currently reading the dml script MNIST LeNet example and got some

questions. I hope that you could help me out of them.

1) Is it possible to define a matrix containing the variables? Because I'm

wondering how to represent the model as a parameter for the "paramserv"

function.

2) What is the role of bias? Why we need it?

Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

to get some feedback. Thanks!

Regards,

Guobao

Hi Guobao,

that sounds very good. In general, the "model" refers to the

collection of all weights and bias matrices of a given architecture.

Similar to a classic regression model, we can view the weights as the

"slope", i.e., multiplicative terms, while the biases are the

"intercept", i.e., additive terms that shift the layer output. Both

are subject to training and thus part of the model.

This implies that the number of matrices in the model depends on the

architecture. Hence, we have two choices here: (a) allow for a

variable number of inputs and outputs, or (b) create a struct-like

data type that allows passing the collection of matrices via a single

handle. We've discussed the second option in other contexts as well

because this would also be useful for reducing the number of

parameters passed through function calls. I'm happy to help out

integrating these struct-like data types if needed.

Great to see that you're in the process of updating the related JIRAs.

Let us know whenever you think you're ready with an initial draft -

then I'd make a detailed pass over it.

Furthermore, I would recommend to experiment with running these

existing mnist lenet examples (which is one of our baselines moving

forward):

* Download the "infinite MNIST" data generator

(http://leon.bottou.org/projects/infimnist), and generate a moderately

sized dataset (e.g., 256K instances).

* Convert the input into SystemML's binary block format. The generator

produces the data in libsvm format and we provide a data converter

(see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

internal binary representation.

* Run the basic mnist lenet example for a few epochs.

* Install the native BLAS libraries mkl or openblas and try using it

for the above example to ensure its setup and configured correctly.

Regards,

Matthias

On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

> Hi Matthias,

>

> I'm currently reading the dml script MNIST LeNet example and got some

> questions. I hope that you could help me out of them.

>

> 1) Is it possible to define a matrix containing the variables? Because I'm

> wondering how to represent the model as a parameter for the "paramserv"

> function.

> 2) What is the role of bias? Why we need it?

>

> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

> to get some feedback. Thanks!

>

> Regards,

> Guobao

>

that sounds very good. In general, the "model" refers to the

collection of all weights and bias matrices of a given architecture.

Similar to a classic regression model, we can view the weights as the

"slope", i.e., multiplicative terms, while the biases are the

"intercept", i.e., additive terms that shift the layer output. Both

are subject to training and thus part of the model.

This implies that the number of matrices in the model depends on the

architecture. Hence, we have two choices here: (a) allow for a

variable number of inputs and outputs, or (b) create a struct-like

data type that allows passing the collection of matrices via a single

handle. We've discussed the second option in other contexts as well

because this would also be useful for reducing the number of

parameters passed through function calls. I'm happy to help out

integrating these struct-like data types if needed.

Great to see that you're in the process of updating the related JIRAs.

Let us know whenever you think you're ready with an initial draft -

then I'd make a detailed pass over it.

Furthermore, I would recommend to experiment with running these

existing mnist lenet examples (which is one of our baselines moving

forward):

* Download the "infinite MNIST" data generator

(http://leon.bottou.org/projects/infimnist), and generate a moderately

sized dataset (e.g., 256K instances).

* Convert the input into SystemML's binary block format. The generator

produces the data in libsvm format and we provide a data converter

(see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

internal binary representation.

* Run the basic mnist lenet example for a few epochs.

* Install the native BLAS libraries mkl or openblas and try using it

for the above example to ensure its setup and configured correctly.

Regards,

Matthias

On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

> Hi Matthias,

>

> I'm currently reading the dml script MNIST LeNet example and got some

> questions. I hope that you could help me out of them.

>

> 1) Is it possible to define a matrix containing the variables? Because I'm

> wondering how to represent the model as a parameter for the "paramserv"

> function.

> 2) What is the role of bias? Why we need it?

>

> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

> to get some feedback. Thanks!

>

> Regards,

> Guobao

>

Hi Matthias,

Thanks for the tips and I'm working on it. So do we have a weekly call at

9:30 08/05 (PST)?

Regards,

Guobao

---------- Forwarded message ----------

From: Matthias Boehm <[EMAIL PROTECTED]>

Date: 2018-05-06 22:08 GMT+02:00

Subject: Re: Questions about MNIST LeNet example

To: [EMAIL PROTECTED]

Cc: Guobao Li <[EMAIL PROTECTED]>

Hi Guobao,

that sounds very good. In general, the "model" refers to the

collection of all weights and bias matrices of a given architecture.

Similar to a classic regression model, we can view the weights as the

"slope", i.e., multiplicative terms, while the biases are the

"intercept", i.e., additive terms that shift the layer output. Both

are subject to training and thus part of the model.

This implies that the number of matrices in the model depends on the

architecture. Hence, we have two choices here: (a) allow for a

variable number of inputs and outputs, or (b) create a struct-like

data type that allows passing the collection of matrices via a single

handle. We've discussed the second option in other contexts as well

because this would also be useful for reducing the number of

parameters passed through function calls. I'm happy to help out

integrating these struct-like data types if needed.

Great to see that you're in the process of updating the related JIRAs.

Let us know whenever you think you're ready with an initial draft -

then I'd make a detailed pass over it.

Furthermore, I would recommend to experiment with running these

existing mnist lenet examples (which is one of our baselines moving

forward):

* Download the "infinite MNIST" data generator

(http://leon.bottou.org/projects/infimnist), and generate a moderately

sized dataset (e.g., 256K instances).

* Convert the input into SystemML's binary block format. The generator

produces the data in libsvm format and we provide a data converter

(see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

internal binary representation.

* Run the basic mnist lenet example for a few epochs.

* Install the native BLAS libraries mkl or openblas and try using it

for the above example to ensure its setup and configured correctly.

Regards,

Matthias

On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

> Hi Matthias,

>

> I'm currently reading the dml script MNIST LeNet example and got some

> questions. I hope that you could help me out of them.

>

> 1) Is it possible to define a matrix containing the variables? Because I'm

> wondering how to represent the model as a parameter for the "paramserv"

> function.

> 2) What is the role of bias? Why we need it?

>

> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

> to get some feedback. Thanks!

>

> Regards,

> Guobao

>

Thanks for the tips and I'm working on it. So do we have a weekly call at

9:30 08/05 (PST)?

Regards,

Guobao

---------- Forwarded message ----------

From: Matthias Boehm <[EMAIL PROTECTED]>

Date: 2018-05-06 22:08 GMT+02:00

Subject: Re: Questions about MNIST LeNet example

To: [EMAIL PROTECTED]

Cc: Guobao Li <[EMAIL PROTECTED]>

Hi Guobao,

that sounds very good. In general, the "model" refers to the

collection of all weights and bias matrices of a given architecture.

Similar to a classic regression model, we can view the weights as the

"slope", i.e., multiplicative terms, while the biases are the

"intercept", i.e., additive terms that shift the layer output. Both

are subject to training and thus part of the model.

This implies that the number of matrices in the model depends on the

architecture. Hence, we have two choices here: (a) allow for a

variable number of inputs and outputs, or (b) create a struct-like

data type that allows passing the collection of matrices via a single

handle. We've discussed the second option in other contexts as well

because this would also be useful for reducing the number of

parameters passed through function calls. I'm happy to help out

integrating these struct-like data types if needed.

Great to see that you're in the process of updating the related JIRAs.

Let us know whenever you think you're ready with an initial draft -

then I'd make a detailed pass over it.

Furthermore, I would recommend to experiment with running these

existing mnist lenet examples (which is one of our baselines moving

forward):

* Download the "infinite MNIST" data generator

(http://leon.bottou.org/projects/infimnist), and generate a moderately

sized dataset (e.g., 256K instances).

* Convert the input into SystemML's binary block format. The generator

produces the data in libsvm format and we provide a data converter

(see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

internal binary representation.

* Run the basic mnist lenet example for a few epochs.

* Install the native BLAS libraries mkl or openblas and try using it

for the above example to ensure its setup and configured correctly.

Regards,

Matthias

On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

> Hi Matthias,

>

> I'm currently reading the dml script MNIST LeNet example and got some

> questions. I hope that you could help me out of them.

>

> 1) Is it possible to define a matrix containing the variables? Because I'm

> wondering how to represent the model as a parameter for the "paramserv"

> function.

> 2) What is the role of bias? Why we need it?

>

> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

> to get some feedback. Thanks!

>

> Regards,

> Guobao

>

just FYI: we now have support for list and named-list data types in

SystemML, which allow passing the entire model as a single handle. For

example, you can define the following

l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or

l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)

and access entries via l1[7] or l2['g'] accordingly. We're still

working on additional features to make the integration with IPA,

functions, and size/type propagation smoother, but the basic

functionality is already available.

Regards,

Matthias

On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <[EMAIL PROTECTED]> wrote:

> Hi Guobao,

>

> that sounds very good. In general, the "model" refers to the

> collection of all weights and bias matrices of a given architecture.

> Similar to a classic regression model, we can view the weights as the

> "slope", i.e., multiplicative terms, while the biases are the

> "intercept", i.e., additive terms that shift the layer output. Both

> are subject to training and thus part of the model.

>

> This implies that the number of matrices in the model depends on the

> architecture. Hence, we have two choices here: (a) allow for a

> variable number of inputs and outputs, or (b) create a struct-like

> data type that allows passing the collection of matrices via a single

> handle. We've discussed the second option in other contexts as well

> because this would also be useful for reducing the number of

> parameters passed through function calls. I'm happy to help out

> integrating these struct-like data types if needed.

>

> Great to see that you're in the process of updating the related JIRAs.

> Let us know whenever you think you're ready with an initial draft -

> then I'd make a detailed pass over it.

>

> Furthermore, I would recommend to experiment with running these

> existing mnist lenet examples (which is one of our baselines moving

> forward):

> * Download the "infinite MNIST" data generator

> (http://leon.bottou.org/projects/infimnist), and generate a moderately

> sized dataset (e.g., 256K instances).

> * Convert the input into SystemML's binary block format. The generator

> produces the data in libsvm format and we provide a data converter

> (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

> internal binary representation.

> * Run the basic mnist lenet example for a few epochs.

> * Install the native BLAS libraries mkl or openblas and try using it

> for the above example to ensure its setup and configured correctly.

>

>

> Regards,

> Matthias

>

> On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

>> Hi Matthias,

>>

>> I'm currently reading the dml script MNIST LeNet example and got some

>> questions. I hope that you could help me out of them.

>>

>> 1) Is it possible to define a matrix containing the variables? Because I'm

>> wondering how to represent the model as a parameter for the "paramserv"

>> function.

>> 2) What is the role of bias? Why we need it?

>>

>> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

>> to get some feedback. Thanks!

>>

>> Regards,

>> Guobao

>>

SystemML, which allow passing the entire model as a single handle. For

example, you can define the following

l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or

l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)

and access entries via l1[7] or l2['g'] accordingly. We're still

working on additional features to make the integration with IPA,

functions, and size/type propagation smoother, but the basic

functionality is already available.

Regards,

Matthias

On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <[EMAIL PROTECTED]> wrote:

> Hi Guobao,

>

> that sounds very good. In general, the "model" refers to the

> collection of all weights and bias matrices of a given architecture.

> Similar to a classic regression model, we can view the weights as the

> "slope", i.e., multiplicative terms, while the biases are the

> "intercept", i.e., additive terms that shift the layer output. Both

> are subject to training and thus part of the model.

>

> This implies that the number of matrices in the model depends on the

> architecture. Hence, we have two choices here: (a) allow for a

> variable number of inputs and outputs, or (b) create a struct-like

> data type that allows passing the collection of matrices via a single

> handle. We've discussed the second option in other contexts as well

> because this would also be useful for reducing the number of

> parameters passed through function calls. I'm happy to help out

> integrating these struct-like data types if needed.

>

> Great to see that you're in the process of updating the related JIRAs.

> Let us know whenever you think you're ready with an initial draft -

> then I'd make a detailed pass over it.

>

> Furthermore, I would recommend to experiment with running these

> existing mnist lenet examples (which is one of our baselines moving

> forward):

> * Download the "infinite MNIST" data generator

> (http://leon.bottou.org/projects/infimnist), and generate a moderately

> sized dataset (e.g., 256K instances).

> * Convert the input into SystemML's binary block format. The generator

> produces the data in libsvm format and we provide a data converter

> (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

> internal binary representation.

> * Run the basic mnist lenet example for a few epochs.

> * Install the native BLAS libraries mkl or openblas and try using it

> for the above example to ensure its setup and configured correctly.

>

>

> Regards,

> Matthias

>

> On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]> wrote:

>> Hi Matthias,

>>

>> I'm currently reading the dml script MNIST LeNet example and got some

>> questions. I hope that you could help me out of them.

>>

>> 1) Is it possible to define a matrix containing the variables? Because I'm

>> wondering how to represent the model as a parameter for the "paramserv"

>> function.

>> 2) What is the role of bias? Why we need it?

>>

>> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope

>> to get some feedback. Thanks!

>>

>> Regards,

>> Guobao

>>

Thanks Matthias! It will be great for passing the model to the paramserv

function.

Regards,

Guobao

2018-05-10 21:47 GMT+02:00 Matthias Boehm <[EMAIL PROTECTED]>:

> just FYI: we now have support for list and named-list data types in

> SystemML, which allow passing the entire model as a single handle. For

> example, you can define the following

>

> l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or

> l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)

>

> and access entries via l1[7] or l2['g'] accordingly. We're still

> working on additional features to make the integration with IPA,

> functions, and size/type propagation smoother, but the basic

> functionality is already available.

>

> Regards,

> Matthias

>

> On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <[EMAIL PROTECTED]> wrote:

> > Hi Guobao,

> >

> > that sounds very good. In general, the "model" refers to the

> > collection of all weights and bias matrices of a given architecture.

> > Similar to a classic regression model, we can view the weights as the

> > "slope", i.e., multiplicative terms, while the biases are the

> > "intercept", i.e., additive terms that shift the layer output. Both

> > are subject to training and thus part of the model.

> >

> > This implies that the number of matrices in the model depends on the

> > architecture. Hence, we have two choices here: (a) allow for a

> > variable number of inputs and outputs, or (b) create a struct-like

> > data type that allows passing the collection of matrices via a single

> > handle. We've discussed the second option in other contexts as well

> > because this would also be useful for reducing the number of

> > parameters passed through function calls. I'm happy to help out

> > integrating these struct-like data types if needed.

> >

> > Great to see that you're in the process of updating the related JIRAs.

> > Let us know whenever you think you're ready with an initial draft -

> > then I'd make a detailed pass over it.

> >

> > Furthermore, I would recommend to experiment with running these

> > existing mnist lenet examples (which is one of our baselines moving

> > forward):

> > * Download the "infinite MNIST" data generator

> > (http://leon.bottou.org/projects/infimnist), and generate a moderately

> > sized dataset (e.g., 256K instances).

> > * Convert the input into SystemML's binary block format. The generator

> > produces the data in libsvm format and we provide a data converter

> > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

> > internal binary representation.

> > * Run the basic mnist lenet example for a few epochs.

> > * Install the native BLAS libraries mkl or openblas and try using it

> > for the above example to ensure its setup and configured correctly.

> >

> >

> > Regards,

> > Matthias

> >

> > On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]>

> wrote:

> >> Hi Matthias,

> >>

> >> I'm currently reading the dml script MNIST LeNet example and got some

> >> questions. I hope that you could help me out of them.

> >>

> >> 1) Is it possible to define a matrix containing the variables? Because

> I'm

> >> wondering how to represent the model as a parameter for the "paramserv"

> >> function.

> >> 2) What is the role of bias? Why we need it?

> >>

> >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and

> hope

> >> to get some feedback. Thanks!

> >>

> >> Regards,

> >> Guobao

> >>

>

function.

Regards,

Guobao

2018-05-10 21:47 GMT+02:00 Matthias Boehm <[EMAIL PROTECTED]>:

> just FYI: we now have support for list and named-list data types in

> SystemML, which allow passing the entire model as a single handle. For

> example, you can define the following

>

> l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or

> l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4)

>

> and access entries via l1[7] or l2['g'] accordingly. We're still

> working on additional features to make the integration with IPA,

> functions, and size/type propagation smoother, but the basic

> functionality is already available.

>

> Regards,

> Matthias

>

> On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <[EMAIL PROTECTED]> wrote:

> > Hi Guobao,

> >

> > that sounds very good. In general, the "model" refers to the

> > collection of all weights and bias matrices of a given architecture.

> > Similar to a classic regression model, we can view the weights as the

> > "slope", i.e., multiplicative terms, while the biases are the

> > "intercept", i.e., additive terms that shift the layer output. Both

> > are subject to training and thus part of the model.

> >

> > This implies that the number of matrices in the model depends on the

> > architecture. Hence, we have two choices here: (a) allow for a

> > variable number of inputs and outputs, or (b) create a struct-like

> > data type that allows passing the collection of matrices via a single

> > handle. We've discussed the second option in other contexts as well

> > because this would also be useful for reducing the number of

> > parameters passed through function calls. I'm happy to help out

> > integrating these struct-like data types if needed.

> >

> > Great to see that you're in the process of updating the related JIRAs.

> > Let us know whenever you think you're ready with an initial draft -

> > then I'd make a detailed pass over it.

> >

> > Furthermore, I would recommend to experiment with running these

> > existing mnist lenet examples (which is one of our baselines moving

> > forward):

> > * Download the "infinite MNIST" data generator

> > (http://leon.bottou.org/projects/infimnist), and generate a moderately

> > sized dataset (e.g., 256K instances).

> > * Convert the input into SystemML's binary block format. The generator

> > produces the data in libsvm format and we provide a data converter

> > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our

> > internal binary representation.

> > * Run the basic mnist lenet example for a few epochs.

> > * Install the native BLAS libraries mkl or openblas and try using it

> > for the above example to ensure its setup and configured correctly.

> >

> >

> > Regards,

> > Matthias

> >

> > On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[EMAIL PROTECTED]>

> wrote:

> >> Hi Matthias,

> >>

> >> I'm currently reading the dml script MNIST LeNet example and got some

> >> questions. I hope that you could help me out of them.

> >>

> >> 1) Is it possible to define a matrix containing the variables? Because

> I'm

> >> wondering how to represent the model as a parameter for the "paramserv"

> >> function.

> >> 2) What is the role of bias? Why we need it?

> >>

> >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and

> hope

> >> to get some feedback. Thanks!

> >>

> >> Regards,

> >> Guobao

> >>

>

Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.

Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.

Service operated by Sematext

Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.

Service operated by Sematext