Questions tagged [adam]

Active Directory Application Mode (ADAM) is an LDAP-compliant directory service.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
14 views

Why is Adam much slower than sgd?

I tried different varieties of gradient descent on a data set. I used gradient descent, stochastic gradient descent , momentum gradient descent , RMSP, Adam etc.. and compared its result. I got Adam ...
0 votes
0 answers
23 views

AttributeError: 'Adam' object has no attribute 'step'

Can anyone assist me in solving this error AttributeError: 'Adam' object has no attribute 'step' When Bert Model is trained with NER, this error occurs. I could not solve this problem, perhaps due to ...
-1 votes
1 answer
28 views

Is Adam optimezer updating weight in every layer?

I'm newbie in Neural network So I little bit confuse about ADAM optimezer. For Example I use MLP with architecture like this: I've used SDG before, so I want to ask if changing the weight with adam's ...
1 vote
0 answers
223 views

Usage of torch.optim.Adam in pytorch

I am new to python and pytorch. I am struggling to understand the usage of Adam optimizer. Please review the below line of code: opt = torch.optim.Adam([y], lr=0.1) This line is taking a input list. ...
  • 25
0 votes
0 answers
25 views

Communicate with Advantech ADAM-6050 through a c++ program

I'm trying to develop a program in c++ that allows reading and setting the logical value of the input and output of the adam 6050. A practical example: I want the code to read the logical value of DI4 ...
0 votes
1 answer
46 views

DIfferent optimization with different TF versions

I'm trying to train a convolutional neural network with keras and Tensorflow version 2.6, also I did it with Tensorflow version 1.11. I think that I did the migration okey (two neural networks ...
0 votes
0 answers
28 views

Is there any solution to replicate multiple Active Directories to the one as a Proxy Active Directory?

We have multiple Active Directories in our company, such as : finance-dc1.company.com finance-dc2.company.com sale-dc1.company.com sale-dc2.company.com development-dc1.company.com development-dc2....
  • 348
42 votes
1 answer
3k views

400% higher error with PyTorch compared with identical Keras model (with Adam optimizer)

TLDR: A simple (single hidden-layer) feed-forward Pytorch model trained to predict the function y = sin(X1) + sin(X2) + ... sin(X10) substantially underperforms an identical model built/trained with ...
  • 1,654
0 votes
0 answers
413 views

'Adam' object has no attribute '_warned_capturable_if_run_uncaptured'

I got above error when I tried to reload the a object of Class for Deep Q Network with target network along with experience replay and train it again. While there are few similar errors related with ...
0 votes
0 answers
53 views

How to create a CoilChange event on input change from hardward device

I am creating a Driver for an Adams Modbus device. When communicating between devices where I have a have an client and a server (TCP based) from which can call events. For example if I have 4 clients ...
  • 2,219
0 votes
0 answers
70 views

Tensorflow SparseCategoricalCrossEntropy loss and Pytorch CrossEntropyLoss and adam optimizer

I'm trying to reproduce the training between tensorflow and pytorch. I came with a simple model using only one linear layer and the dataset that I'm using is the mnist hand digit. Before testing I ...
0 votes
0 answers
38 views

TypeError: object() takes no parameters | when executing optimizer.apply_gradients(zip(grad,model.trainable_weights))

Here is the entire code for the execution of the class below. class RML_model: def __init__(self,K): self.model=Policy_Network_Model() self.prev_eval=0.0 self.K=K # number of samples ...
0 votes
1 answer
70 views

Pythorch: Dictionary learning with Neural Networks. Adam Opt does not converge

I have implemented a code based on the paper of https://arxiv.org/pdf/1707.00225.pdf However, the algorithm 1 of the paper (which is basically 2 steps: first step you calculate K with the parameters ...
  • 21
0 votes
0 answers
51 views

How can I add SSIM?

I'm trying to add an SSIM to the following code and print the results in a table at the end, but I just can't seem to make it work, can you help me? Also my adam optimizer is not working I have no ...
1 vote
0 answers
77 views

How to save Adam optimizer parameters in Keras?

Adam algorithm in Keras: Initialize m_0 as 1st-moment vector, Initialize v_0as 2nd-moment vector, The update rule for theta with gradient g: lr_t = learning_rate * sqrt(1 - beta_2^t) / (1 - beta_1^t) ...
  • 125
0 votes
1 answer
3k views

AttributeError: module 'keras.optimizers' has no attribute 'Adam' , I am getting this error

I am getting this error AttributeError: module 'keras.optimizers' has no attribute 'Adam' for the below. classifier.compile(optimizer= keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0....
0 votes
1 answer
913 views

How do we have access to the effective learning rate of Adam [Tensorflow]?

I am interested in the effective learning rate of Adam. We know that Adam is roughly formed by a initial/constant learning rate divided by tthe sum of the past gradients of the loss (see here for ...
  • 174
0 votes
0 answers
17 views

The Loss number of both train set and validation set both went down at the begining and then went up again

I try to use the siamese network which combines with two resnet networks (Pretrained) to solve the few shot problem. The loss function is contrassive loss and the optimizer is Adam with 0.001 learning ...
1 vote
2 answers
2k views

How to solve the problem with tf.keras.optimizers.Adam(lr=0.001) command not working?

I'm working on Google Colab and when I type model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-6), loss=tf.keras.losses.BinaryCrossentropy()) it doesn't work and I get the following error message ...
0 votes
1 answer
1k views

'Adam' object has no attribute 'Adam'

This is how I imported the modules from keras.layers import Conv2D, BatchNormalization, Activation from keras.models import Model, Input from keras import optimizer_v2 from keras.optimizer_v2 import ...
  • 23
0 votes
0 answers
31 views

hello, I need a help to connect my spyder source code with excel sheet. first problem

enter image description here The future prediction part of the graph is strange. It's not predictable at all. The second question is, I want to import the data of the predicted part into an Excel file....
0 votes
1 answer
271 views

Adam Optimizer not Updating Values

I am trying to use Adam optimizer to obtain certain values outside of a neural network. My technique wasn't working so I created a simple example to see if it works: a = np.array([[0.0,1.0,2.0,3.0,4.0]...
0 votes
0 answers
207 views

Custom Optimizer error--> TypeError: Expected tf.group() expected Tensor arguments not 'None' with type '<class 'NoneType'>'

I've implemented an accumulated gradient optimizer but when I want to train model it gives me this error.So what is the problem? The idea behind gradient accumulation is that it calculates the loss ...
0 votes
1 answer
1k views

Node 'training/Adam/gradients/gradients/conv5_block3_3_bn/cond_grad/StatelessIf'

img_height,img_width = 32, 32 base_model = ResNet50(weights = 'imagenet', include_top = False, input_shape =(img_height,img_width,3)) x = base_model.output x = GlobalAveragePooling2D()(x) x = Dropout(...
0 votes
0 answers
535 views

Adam optimization for gradient descent update doesn't seem to work with logistic regression

Helo, I'm learning machine learning from first principle, so i coded up logistic regression with back prop from scratch using numpy and calculus. Updating derivative with weighted average (momentum) ...
0 votes
1 answer
414 views

Tensorflow Adam Optimizer state not updating ( get_config )

I am using optimizer.get_config() to get the final state of my adam optimizer (as in https://stackoverflow.com/a/60077159/607528) however .get_config() is returning the initial state. I assume this ...
  • 247
2 votes
0 answers
219 views

Adam custom implementation by PyTorch

I'm trying to code my own implementation of Adam optimization algorithm, but when I try to find the optimum for function f(x,y) = xx + yy, method generates an unexpected output. Here is the code and ...
0 votes
1 answer
83 views

Using Netscape library for performing LDAP search operation and getting limited result upto 10000 when range is provided (0-*)

I am using Netscape library for performing search operation on microsoft ADS/ADAM Ldap server Following is the snippet I am using: LDAPConnection connection=new LDAPConnection(); connection.connect(&...
  • 5
2 votes
0 answers
498 views

Using Adam to find the minimum of the Rosenbrock function using Pytorch

I am comparing the Adam - Algorithm to SGD with Momentum. I realised that the convergence rate of Adam is way worse than the convergence rate of SGD with Momentum if applied to the Rosenbrock function....
0 votes
1 answer
100 views

Error while training logistic regression on FashionMNIST with Adam optimizer

Dataset is FashionMNIST (784 input, 10 output). I'm trying to train logistic regression with Adam optimizer (coded it also): weights = torch.randn(784, 10) / math.sqrt(784) weights.requires_grad_() ...
0 votes
1 answer
50 views

When is Momentum Applied in Tensorflow Gradient Tape?

I've been playing around with automatic gradients in tensorflow and I had a question. If we are updating an optimizer, say ADAM, when is the momentum algorithm applied to the gradient? Is it applied ...
  • 1
1 vote
0 answers
81 views

Is there an implementation of Keras Adam Optimizer to support Float16

I am currently working with deploying tiny-yolov3 on openvino toolkit and for that i need to convert my model to float16. But for that I need an optimizer that supports FP16. I tried modifying SGD to ...
1 vote
1 answer
421 views

Why does my learning rate decrease, even when loss is improving?

I am training my Keras model on Google Colab TPU as follows - adam = Adam(lr=0.002) model.compile(loss='mse', metrics=[PSNRLoss, SSIMLoss], optimizer=adam) checkpoint = ModelCheckpoint("model_{...
0 votes
0 answers
604 views

Having problem with creating a dataloader class in PyTorch

I am building a neural network for Bengali numerical digit classification using PyTorch. I am facing difficulties building the dataset class to load my dataset using a data loader. I have a folder ...
1 vote
1 answer
861 views

How to use Lazy Adam optimizer in tensorflow 2.0.0

This code doesnt work: it has problem with tf.contrib model.compile(optimizer=TFOptimizer(tf.contrib.opt.LazyAdamOptimizer()), loss='categorical_crossentropy') I have tried something with ...
  • 640
2 votes
1 answer
4k views

Is Adam optimizer really RMSprop plus momentum? If yes, why it doesn't have a momentum parameter?

Here is a link to tensorflow optimizers. And there you can see, that RMSprop takes momentum as argument while Adam does not do this. So I am confuzed. Adam optimization pretends to be a RMSprop ...
  • 1,714
1 vote
1 answer
1k views

5 epoch too soon for a model to converge?

I'm trying to build a CNN model, I've about 44,000 images with 6 class, image size being 99X99. First i tried ResNet10 using SGD optimizer, i set it for 30 epochs with learning rate of 0.001 and the ...
2 votes
1 answer
742 views

can't reproduce model.fit with GradientTape

I've been trying to investigate into the reason (e.g. by checking weights, gradients and activations during training) why SGD with a 0.001 learning rate worked in training while Adam fails to do so. (...
  • 65
3 votes
1 answer
866 views

Resume training with Adam optimizer in Keras

My question is quite straightforward but I can't find a definite answer online (so far). I have saved the weights of a keras model trained with an adam optimizer after a defined number of epochs of ...
  • 67
0 votes
0 answers
2k views

Getting nan loss in Adam Model using PyTorch

I am new to training neural nets. Please forgive me if this is a very stupid question or violates any of the unsaid rules of stack overflow. I started working on the titanic data set recently. I ...
4 votes
2 answers
4k views

epsilon parameter in Adam opitmizer

Using pyTorch and tensorflow (TF), I was wandering how the Adam optimizer is implemented for curiosity. And I do not know if I am wrong or not but it seems to me that the two implementations differ ...
  • 363
1 vote
0 answers
262 views

Does Keras Adam Optimizer and other momemtum-based optimizers retain its past update information over different fit calls?

The Adam optimizer uses a momentum-like approach to train a neural network in fewer iterations of gradient descent than vanilla gradient descent. I'm trying to figure out whether the Adam optimizer ...
3 votes
1 answer
924 views

Why does the learning rate increase in Adam?

I have been using the following piece of code to print the lr_t learning_rate in Adam() optimizer for my trainable_model. if(np.random.uniform()*100 < 3 and self.training): model = self....
  • 95
2 votes
1 answer
750 views

Model learns with SGD but not Adam

I was going through a basic PyTorch MNIST example here and noticed that when I changed the optimizer from SGD to Adam the model did not converge. Specifically, I changed line 106 from optimizer = ...
0 votes
1 answer
374 views

Why is Adam's get_update only executed once?

Why get_update() is not executed every epochs? myAdam: @interfaces.legacy_get_updates_support def get_updates(self, loss, params): print("update!") #Other code is the same ...
  • 3
0 votes
0 answers
234 views

How to update AD LDS distinguishedName attribute to Single Value to No

Currently the Single Value is set to Yes. How do i modify it ? Why I am trying to do this is because I want to have a unique DN for each person. Let say I have 2 person name is cn=Alice, i can do a ...
0 votes
1 answer
50 views

Where is this dimension error getting generated in my ADAM algorithm?

I am working on main.py in this BRATS Unet https://github.com/pykao/Modified-3D-UNet-Pytorch/blob/master/main.py # create your optimizer print ("Creating Optimizer") ##optimizer = optim.adam(net....
  • 423
2 votes
1 answer
227 views

Access optimizers internal state

I am using the DQN Agent from Ray/RLLib. To gain more insight into how the training process is going, I would like to access the internal state of the Adam-Optimizer, to eg visualize how the running ...
0 votes
1 answer
59 views

2sxc Adam folder list

I have a problem with listing/showing Folders created within Adam's File Library? I have dowloaded Adam 2.0 tutorial from your site I can't get them listed even in downloaded template and example "...
3 votes
0 answers
621 views

Using custom optimizer in Keras after loading model

I would like to use an AdamW optimizer in keras for NN training, according to: https://github.com/GLambard/AdamW_Keras I import it and the code appears to work as advertised. Unfortunately, when I ...