Questions tagged [adam]
Active Directory Application Mode (ADAM) is an LDAP-compliant directory service.
158
questions
-1
votes
0
answers
14
views
Why is Adam much slower than sgd?
I tried different varieties of gradient descent on a data set. I used gradient descent, stochastic gradient descent , momentum gradient descent , RMSP, Adam etc.. and compared its result. I got Adam ...
0
votes
0
answers
23
views
AttributeError: 'Adam' object has no attribute 'step'
Can anyone assist me in solving this error AttributeError: 'Adam' object has no attribute 'step' When Bert Model is trained with NER, this error occurs. I could not solve this problem, perhaps due to ...
-1
votes
1
answer
28
views
Is Adam optimezer updating weight in every layer?
I'm newbie in Neural network So I little bit confuse about ADAM optimezer. For Example I use MLP with architecture like this:
I've used SDG before, so I want to ask if changing the weight with adam's ...
1
vote
0
answers
223
views
Usage of torch.optim.Adam in pytorch
I am new to python and pytorch. I am struggling to understand the usage of Adam optimizer.
Please review the below line of code:
opt = torch.optim.Adam([y], lr=0.1)
This line is taking a input list. ...
0
votes
0
answers
25
views
Communicate with Advantech ADAM-6050 through a c++ program
I'm trying to develop a program in c++ that allows reading and setting the logical value of the input and output of the adam 6050.
A practical example: I want the code to read the logical value of DI4 ...
0
votes
1
answer
46
views
DIfferent optimization with different TF versions
I'm trying to train a convolutional neural network with keras and Tensorflow version 2.6, also I did it with Tensorflow version 1.11. I think that I did the migration okey (two neural networks ...
0
votes
0
answers
28
views
Is there any solution to replicate multiple Active Directories to the one as a Proxy Active Directory?
We have multiple Active Directories in our company, such as :
finance-dc1.company.com
finance-dc2.company.com
sale-dc1.company.com
sale-dc2.company.com
development-dc1.company.com
development-dc2....
42
votes
1
answer
3k
views
400% higher error with PyTorch compared with identical Keras model (with Adam optimizer)
TLDR:
A simple (single hidden-layer) feed-forward Pytorch model trained to predict the function y = sin(X1) + sin(X2) + ... sin(X10) substantially underperforms an identical model built/trained with ...
0
votes
0
answers
413
views
'Adam' object has no attribute '_warned_capturable_if_run_uncaptured'
I got above error when I tried to reload the a object of Class for Deep Q Network with target network along with experience replay and train it again.
While there are few similar errors related with ...
0
votes
0
answers
53
views
How to create a CoilChange event on input change from hardward device
I am creating a Driver for an Adams Modbus device. When communicating between devices where I have a have an client and a server (TCP based) from which can call events.
For example if I have 4 clients ...
0
votes
0
answers
70
views
Tensorflow SparseCategoricalCrossEntropy loss and Pytorch CrossEntropyLoss and adam optimizer
I'm trying to reproduce the training between tensorflow and pytorch. I came with a simple model using only one linear layer and the dataset that I'm using is the mnist hand digit. Before testing I ...
0
votes
0
answers
38
views
TypeError: object() takes no parameters | when executing optimizer.apply_gradients(zip(grad,model.trainable_weights))
Here is the entire code for the execution of the class below.
class RML_model:
def __init__(self,K):
self.model=Policy_Network_Model()
self.prev_eval=0.0
self.K=K # number of samples ...
0
votes
1
answer
70
views
Pythorch: Dictionary learning with Neural Networks. Adam Opt does not converge
I have implemented a code based on the paper of https://arxiv.org/pdf/1707.00225.pdf
However, the algorithm 1 of the paper (which is basically 2 steps: first step you calculate K with the parameters ...
0
votes
0
answers
51
views
How can I add SSIM?
I'm trying to add an SSIM to the following code and print the results in a table at the end, but I just can't seem to make it work, can you help me?
Also my adam optimizer is not working I have no ...
1
vote
0
answers
77
views
How to save Adam optimizer parameters in Keras?
Adam algorithm in Keras:
Initialize m_0 as 1st-moment vector,
Initialize v_0as 2nd-moment vector,
The update rule for theta with gradient g:
lr_t = learning_rate * sqrt(1 - beta_2^t) / (1 - beta_1^t)
...
0
votes
1
answer
3k
views
AttributeError: module 'keras.optimizers' has no attribute 'Adam' , I am getting this error
I am getting this error
AttributeError: module 'keras.optimizers' has no attribute 'Adam' for the below.
classifier.compile(optimizer= keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0....
0
votes
1
answer
913
views
How do we have access to the effective learning rate of Adam [Tensorflow]?
I am interested in the effective learning rate of Adam. We know that Adam is roughly formed by a initial/constant learning rate divided by tthe sum of the past gradients of the loss (see here for ...
0
votes
0
answers
17
views
The Loss number of both train set and validation set both went down at the begining and then went up again
I try to use the siamese network which combines with two resnet networks (Pretrained) to solve the few shot problem. The loss function is contrassive loss and the optimizer is Adam with 0.001 learning ...
1
vote
2
answers
2k
views
How to solve the problem with tf.keras.optimizers.Adam(lr=0.001) command not working?
I'm working on Google Colab and when I type
model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-6), loss=tf.keras.losses.BinaryCrossentropy())
it doesn't work and I get the following error message
...
0
votes
1
answer
1k
views
'Adam' object has no attribute 'Adam'
This is how I imported the modules
from keras.layers import Conv2D, BatchNormalization, Activation
from keras.models import Model, Input
from keras import optimizer_v2
from keras.optimizer_v2 import ...
0
votes
0
answers
31
views
hello, I need a help to connect my spyder source code with excel sheet. first problem
enter image description here
The future prediction part of the graph is strange. It's not predictable at all. The second question is, I want to import the data of the predicted part into an Excel file....
0
votes
1
answer
271
views
Adam Optimizer not Updating Values
I am trying to use Adam optimizer to obtain certain values outside of a neural network. My technique wasn't working so I created a simple example to see if it works:
a = np.array([[0.0,1.0,2.0,3.0,4.0]...
0
votes
0
answers
207
views
Custom Optimizer error--> TypeError: Expected tf.group() expected Tensor arguments not 'None' with type '<class 'NoneType'>'
I've implemented an accumulated gradient optimizer but when I want to train model it gives me this error.So what is the problem?
The idea behind gradient accumulation is that it calculates the loss ...
0
votes
1
answer
1k
views
Node 'training/Adam/gradients/gradients/conv5_block3_3_bn/cond_grad/StatelessIf'
img_height,img_width = 32, 32
base_model = ResNet50(weights = 'imagenet', include_top = False, input_shape =(img_height,img_width,3))
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dropout(...
0
votes
0
answers
535
views
Adam optimization for gradient descent update doesn't seem to work with logistic regression
Helo, I'm learning machine learning from first principle, so i coded up logistic regression with back prop from scratch using numpy and calculus. Updating derivative with weighted average (momentum) ...
0
votes
1
answer
414
views
Tensorflow Adam Optimizer state not updating ( get_config )
I am using optimizer.get_config() to get the final state of my adam optimizer (as in https://stackoverflow.com/a/60077159/607528) however .get_config() is returning the initial state. I assume this ...
2
votes
0
answers
219
views
Adam custom implementation by PyTorch
I'm trying to code my own implementation of Adam optimization algorithm, but when I try to find the optimum for function f(x,y) = xx + yy, method generates an unexpected output.
Here is the code and ...
0
votes
1
answer
83
views
Using Netscape library for performing LDAP search operation and getting limited result upto 10000 when range is provided (0-*)
I am using Netscape library for performing search operation on microsoft ADS/ADAM Ldap server
Following is the snippet I am using:
LDAPConnection connection=new LDAPConnection();
connection.connect(&...
2
votes
0
answers
498
views
Using Adam to find the minimum of the Rosenbrock function using Pytorch
I am comparing the Adam - Algorithm to SGD with Momentum. I realised that the convergence rate of Adam is way worse than the convergence rate of SGD with Momentum if applied to the Rosenbrock function....
0
votes
1
answer
100
views
Error while training logistic regression on FashionMNIST with Adam optimizer
Dataset is FashionMNIST (784 input, 10 output). I'm trying to train logistic regression with Adam optimizer (coded it also):
weights = torch.randn(784, 10) / math.sqrt(784)
weights.requires_grad_()
...
0
votes
1
answer
50
views
When is Momentum Applied in Tensorflow Gradient Tape?
I've been playing around with automatic gradients in tensorflow and I had a question. If we are updating an optimizer, say ADAM, when is the momentum algorithm applied to the gradient? Is it applied ...
1
vote
0
answers
81
views
Is there an implementation of Keras Adam Optimizer to support Float16
I am currently working with deploying tiny-yolov3 on openvino toolkit and for that i need to convert my model to float16. But for that I need an optimizer that supports FP16. I tried modifying SGD to ...
1
vote
1
answer
421
views
Why does my learning rate decrease, even when loss is improving?
I am training my Keras model on Google Colab TPU as follows -
adam = Adam(lr=0.002)
model.compile(loss='mse', metrics=[PSNRLoss, SSIMLoss], optimizer=adam)
checkpoint = ModelCheckpoint("model_{...
0
votes
0
answers
604
views
Having problem with creating a dataloader class in PyTorch
I am building a neural network for Bengali numerical digit classification using PyTorch. I am facing difficulties building the dataset class to load my dataset using a data loader. I have a folder ...
1
vote
1
answer
861
views
How to use Lazy Adam optimizer in tensorflow 2.0.0
This code doesnt work: it has problem with tf.contrib
model.compile(optimizer=TFOptimizer(tf.contrib.opt.LazyAdamOptimizer()), loss='categorical_crossentropy')
I have tried something with ...
2
votes
1
answer
4k
views
Is Adam optimizer really RMSprop plus momentum? If yes, why it doesn't have a momentum parameter?
Here is a link to tensorflow optimizers. And there you can see, that RMSprop takes momentum as argument while Adam does not do this. So I am confuzed. Adam optimization pretends to be a RMSprop ...
1
vote
1
answer
1k
views
5 epoch too soon for a model to converge?
I'm trying to build a CNN model, I've about 44,000 images with 6 class, image size being 99X99. First i tried ResNet10 using SGD optimizer, i set it for 30 epochs with learning rate of 0.001 and the ...
2
votes
1
answer
742
views
can't reproduce model.fit with GradientTape
I've been trying to investigate into the reason (e.g. by checking weights, gradients and activations during training) why SGD with a 0.001 learning rate worked in training while Adam fails to do so. (...
3
votes
1
answer
866
views
Resume training with Adam optimizer in Keras
My question is quite straightforward but I can't find a definite answer online (so far).
I have saved the weights of a keras model trained with an adam optimizer after a defined number of epochs of ...
0
votes
0
answers
2k
views
Getting nan loss in Adam Model using PyTorch
I am new to training neural nets. Please forgive me if this is a very stupid question or violates any of the unsaid rules of stack overflow. I started working on the titanic data set recently. I ...
4
votes
2
answers
4k
views
epsilon parameter in Adam opitmizer
Using pyTorch and tensorflow (TF), I was wandering how the Adam optimizer is implemented for curiosity. And I do not know if I am wrong or not but it seems to me that the two implementations differ ...
1
vote
0
answers
262
views
Does Keras Adam Optimizer and other momemtum-based optimizers retain its past update information over different fit calls?
The Adam optimizer uses a momentum-like approach to train a neural network in fewer iterations of gradient descent than vanilla gradient descent. I'm trying to figure out whether the Adam optimizer ...
3
votes
1
answer
924
views
Why does the learning rate increase in Adam?
I have been using the following piece of code to print the lr_t learning_rate in Adam() optimizer for my trainable_model.
if(np.random.uniform()*100 < 3 and self.training):
model = self....
2
votes
1
answer
750
views
Model learns with SGD but not Adam
I was going through a basic PyTorch MNIST example here and noticed that when I changed the optimizer from SGD to Adam the model did not converge. Specifically, I changed line 106 from
optimizer = ...
0
votes
1
answer
374
views
Why is Adam's get_update only executed once?
Why get_update() is not executed every epochs?
myAdam:
@interfaces.legacy_get_updates_support
def get_updates(self, loss, params):
print("update!")
#Other code is the same
...
0
votes
0
answers
234
views
How to update AD LDS distinguishedName attribute to Single Value to No
Currently the Single Value is set to Yes. How do i modify it ?
Why I am trying to do this is because I want to have a unique DN for each person.
Let say I have 2 person name is cn=Alice, i can do a ...
0
votes
1
answer
50
views
Where is this dimension error getting generated in my ADAM algorithm?
I am working on main.py in this BRATS Unet
https://github.com/pykao/Modified-3D-UNet-Pytorch/blob/master/main.py
# create your optimizer
print ("Creating Optimizer")
##optimizer = optim.adam(net....
2
votes
1
answer
227
views
Access optimizers internal state
I am using the DQN Agent from Ray/RLLib. To gain more insight into how the training process is going, I would like to access the internal state of the Adam-Optimizer, to eg visualize how the running ...
0
votes
1
answer
59
views
2sxc Adam folder list
I have a problem with listing/showing Folders created within Adam's File Library?
I have dowloaded Adam 2.0 tutorial from your site
I can't get them listed even in downloaded template and example "...
3
votes
0
answers
621
views
Using custom optimizer in Keras after loading model
I would like to use an AdamW optimizer in keras for NN training, according to:
https://github.com/GLambard/AdamW_Keras
I import it and the code appears to work as advertised. Unfortunately, when I ...