Raju Gautam

It is always better to work on a separate database while upgrading. And also it is wise to do it in local system first for sure rather than putting the site offline during upgrade process. It is not only because we are not sure but Magento does not have its own formal steps to upgrade. So dump the existing old database and import in new database name.

Steps:
I did the following steps to upgrade Magento 1.4.2.0 to 1.6.1.0:

• - Installed fresh 1.6 Magento with fresh sample data (sample data should also be the latest 1.6 version).
• - Re-indexed all the items from back end of installed latest Magento 1.6.
• - Disabled all the cache from Magento back end.
• - Flushed/deleted all the Magento cache.

  rm -rf var/cache/* var/session/*
  rm -rf downloader/pearlib/cache/* downloader/pearlib/download/*
  

• - Prevent unique check while upgrading by changing file app/etc/config.xml

  SET NAMES utf8;

  To

  SET NAMES utf8; SET FOREIGN_KEY_CHECKS=0; SET UNIQUE_CHECKS=0;

• - To trace the errors, I renamed the errors/local.xml.sample to errors/local.xml.
• - Edited the app/etc/local.xml to connect to the old database.
• - I was working in Ubuntu, so in terminal, I ran the following command in the Magento 1.6 installed root directory.
  php index.php
  (If in case of windows just hit the Magento home URL in the browser and wait till it ends)

The above steps are standard steps for all of the Magento upgrades. But troubleshooting may vary depending upon the Magento system, version, extensions used, data, extra attributes added, etc.

Troubleshooting while upgrading:
Problem:
First of all I encountered the error of it failed renaming the core_url_rewrite table. I don’t know why does it really needed to rename it but the error was on renaming it. I tried several times but I always found the same error with “core_url_rewrite” table. I was almost stopped here. There was nothing except pulling the hair out even after a many times of googling.

The error was with that table and I tried by trunctating the table but no luck. Next I tried by dropping the table but again no luck.

Solution:
I droped the table from old table and imported the structure of the table from new 1.6.1.0 database and tried to upgrade again then it worked for me. You do not have to worry about the data in this table because while reindexing, the data will be filled out again.

Problem:
Secondly, there was an error for customer integrity constraint violation.

Solution:
I compared the structure of customer_entity (of new and old database) and found “website_id and email” should be unique in new structure. Checked if there are duplicate entry with those two fields:

SELECT *
FROM `customer_entity`
GROUP BY website_id,email HAVING COUNT(entity_id)>1

Found one email address was duplicate for the same website and I deleted one.

Then again ran the command :

php index.php

It took around 1 hour but this time there was no message. That means my upgrade has been done

Then flushed the cache again!

Change the base URLs in the “core_config_data” table for the path “web/secure/base_url” and “web/unsecure/base_url”.

Reindexing:
Before visiting the site, don’t forget to reindex.

php shell/indexer.php --reindexall

Again flushed the cache.

rm -rf var/cache/* var/session/*
rm -rf downloader/pearlib/cache/* downloader/pearlib/download/*

Then I visited the admin site. I was able to login to the admin with new version of Magento.

I did not stop here. Not to take long breath yet. Now I had to check each and every menus available in Magento back end. All were working fine but product edit page was with fatal error and order detail view page was with broken design but no error found.

After some research and study about modules used in the old database, extra attributes used in old version, I thought I had to delete all the attributes and module related data.

I tried my best to search the modules related data and attributes in the database and remove them. I had removed some attributes, I had updated some attributes which has used models from “eav_attribute” table. And I had to remove the modules setup data from “core_resource” table.

Search:

SELECT * FROM `eav_attribute` WHERE `attribute_code` like 'aw_%';
SELECT * FROM `eav_attribute` WHERE `attribute_code` like 'bcp_%';

Set all the models that attribute calls to NULL and remove the attribute from being System Attribute (`is_user_defined`=1) so that it can be removed from the admin Attribute Manager.

UPDATE `eav_attribute` SET
`attribute_model`=NULL,
`backend_model`=NULL,
`frontend_model`=NULL,
`source_model`=NULL,
`is_user_defined`=1
WHERE `attribute_code` like 'aw_%';

For another extension:

UPDATE `eav_attribute` SET
`attribute_model`=NULL,
`backend_model`=NULL,
`frontend_model`=NULL,
`source_model`=NULL,
`is_user_defined`=1
WHERE `attribute_code` like 'bcp_%';

I also removed those extensions entry from “core_resource” table.

DELETE FROM `core_resource` WHERE `code` = 'awall_setup'
DELETE FROM `core_resource` WHERE `core_resource`.`code` = 'bcp_setup'

Do not forget to flush the cache again before running the site.

rm -rf var/cache/* var/session/*
rm -rf downloader/pearlib/cache/* downloader/pearlib/download/*

So in my case, specially the AW extensions and their attributes were the cause of the error in product listing and order view page.

Hope these steps will help the people who are really desparate to upgrade Magento systems and having trouble. I believe at least some of the steps and troubleshooting tips will help someone.

Enjoy!!

Magento acts always something strange. I struggled around 5/6 hours for fixing this particular small problem.

I migrated successfully a Magento shop from 1.4.2 to latest version 1.6.0. I was trying to move my local files to my Ubuntu system. While migrating I was using a different sub domain and now I am trying some different tasks with different sub domain. I just made a copy of the running site (files and DB) and made requried configurations. And tried to login to the admin with correct username and password, I was just redirected to the same login page without any errors/warnings/messages. When I entere wrong username/password then it shows me the error.

That was quite annoying .

I looked in the google and found one solution to modify (comment some lines of code) in the core file (app/code/core/Mage/Core/Model/Session/Abstract/Varien.php) as follows:

$this->getCookie()->getPath()//,
 // set session cookie params
 session_set_cookie_params(
 $this->getCookie()->getLifetime(),
 $this->getCookie()->getPath()//,
 //$this->getCookie()->getDomain(),
 //$this->getCookie()->isSecure(),
 //$this->getCookie()->getHttponly()
 );

It was written that for 1.4 it has to be something like below:

/* if (!$cookieParams['httponly']) {
   unset($cookieParams['httponly']);
   if (!$cookieParams['secure']) {
     unset($cookieParams['secure']);
     if (!$cookieParams['domain']) {
       unset($cookieParams['domain']);
     }
   }
 }
 if (isset($cookieParams['domain'])) {
   $cookieParams['domain'] = $cookie->getDomain();
 } */

When I comment these lines and tried and it worked. Strange!!!

Then I dig into the Magento core files and tried to find what exactly these lines of code check and found that they check the domain for the cookie stored in the core_config_data table. I saw a row in core_config_data table having the old sub domain with the path ‘web/cookie/cookie_domain’. Then I changed this domain value to my new sub domain and uncomment above lines of code.

Then it worked!! Bravo!!!

Magento always rocks but before rocking it sucks for a while…

Good luck others!!!

Since Magento itself is too vast not only because of its features but also because of heavy number of files within it, it is really difficult to find which file is being used for what purpose and where that file is really located.

Front End:
Magento has provided a feature in its configuration section in admin that allows us to enable template path hints. If we enable it, it will shows the file pathand block names.

To enable it:
1. Login to admin.
2. Go to System->Configuration.
3. Select a website from and click Developer tab in left.
4. In the Debug section, select Yes for Template Path Hints & Add Block Names to Hints.
5. Save configuration and refresh the front end page you will have some red blocks to indicate what files files are used in which block along with Block (class names) used.

Magento Template Path Hints Without Changing in Admin:
But enabling and disabling it from back end is boring in time to time. So I thought why not just enable the hints with some values passed via URL like http://www.mymagentosite.com/?template_hint=true or something like this. I searched where actually Magento handles this and found at app/code/core/Mage/Core/Block/Template.php. There is a method called getShowTemplateHints() at around line number 176 which handles it. The function originally looks like this:

public function getShowTemplateHints()
{
	if (is_null(self::$_showTemplateHints)) {
		self::$_showTemplateHints = Mage::getStoreConfig('dev/debug/template_hints')
			&& Mage::helper('core')->isDevAllowed();
		self::$_showTemplateHintsBlocks = Mage::getStoreConfig('dev/debug/template_hints_blocks')
			&& Mage::helper('core')->isDevAllowed();
	}
	return self::$_showTemplateHints;
}

So the two static members (self::$_showTemplateHints & self::$_showTemplateHintsBlocks) of the class are to set to handle them. So I modified it to be something like this:

public function getShowTemplateHints()
{
	if (is_null(self::$_showTemplateHints)) {
		self::$_showTemplateHints = Mage::getStoreConfig('dev/debug/template_hints')
			&& Mage::helper('core')->isDevAllowed();
		self::$_showTemplateHintsBlocks = Mage::getStoreConfig('dev/debug/template_hints_blocks')
			&& Mage::helper('core')->isDevAllowed();
	}

	// overwrite the template hit
	$th 	= Mage::app()->getRequest()->getParam('th', false);
	$token 	= Mage::app()->getRequest()->getParam('token', false);
	if($th == 1 && $token == 'PHP'){
		self::$_showTemplateHints = true; // for template path
		self::$_showTemplateHintsBlocks = true; // block names
	}

	return self::$_showTemplateHints;
}

I have used two variables for security. Now if ‘th’ and ‘token’ are set with specific values then both of the static members are set to true and hence the templates will be now visible. Now if you hit something like this in your browser http://www.mymagentosite.com/?th=1&token=PHP you can see template hints and added Block Names. I found this quite handy rather than going to admin and enable/disable.

Admin:
For admin, there is no direct feature, so there is a trick direcly modifying the database table and values. This I had found in the internet. You need to add two rows in the table ‘core_config_data’ as follows for the first time:

INSERT INTO core_config_data (scope, scope_id, path, value) VALUES
('default', 0, 'dev/debug/template_hints', 1),
('default', 0, 'dev/debug/template_hints_blocks', 1);

If you have already those rows then you just need to update the ‘value’ field with 1 to enable and 0 to disable.
Disable:

UPDATE core_config_data SET value=0 WHERE scope='default' AND scope_id = 0 AND path ='dev/debug/template_hints'

Enable it again:

UPDATE core_config_data SET value=1 WHERE scope='default' AND scope_id = 0 AND path ='dev/debug/template_hints'

Hope this will help .

Though I am still new and have to learn a lot in Magento but I am struggling for Magento’s some serious issues of our clients. Among the so many problems with working experience in Magento, one of our customer encountered a database table corrupt problem and he asked the technical guy in their server to fix the problem. The technical person just repaired the database and the the Magento front end site worked perfectly fine. But when we try to go to admin of the shop, then it always shows the 404 page instead of showing the Admin login form.

After some research, I came to know that it is something about the store view for admin section which is supposed to be in the ‘core_store’ table. When the database was repaired, two rows were inserted (or updated) as follows:

--------------------------------------------------------------------------
`store_id`,`code`,`website_id`,`group_id`,`name`,`sort_order`,`is_active`
--------------------------------------------------------------------------
'2', 'admin', '0', '5', 'Admin', '0', '1'
'1', 'default', '0', '5', 'Admin', '0', '1'
--------------------------------------------------------------------------

I just compared with another Magento shop and there was ’0′ as stored_id for admin. And I just edited the row and put it as ’0′ instead of ’2′ and saved it.

Now the Admin section worked like a charm !

Strange problem but easy fix (after a research of 3/4 hours) !!

Just shared it hoping it will help others encountering the same problem

Magento has lots of features as an ecommerce application. No doubt that’s why it is too popular too. These days most of the clients (end customers/users) prefer Magento by themselves. But there are some pitfalls within it. Once explore it specialy working as a developer (programmer), you will be in a trouble in such way that you are no more going to use Magento for any purpose because you will stucked in such a problem which neither can be solved nor you can directly change Magento’s code itself. Saying so, I don’t mean that we cannot solve those problems. Indeed we can solve the problems sometimes by hacking the core codes and in the other times by changing some values directly from the database. Since Magento is designed and developed with EAV (Entities, Attributes & Values) model, it is really difficult to find a particular data/value within the database too. And most of the times we extend its features by creating our own modules.

Anyway, here I am going to present a hack to solve a problem.

Problem:
For SEO friendly URLS, Magento has a field for a Product and Category to create custom SEO friendly URL keys. But when you have multiple stores, then you cannot normally have different URL keys for different stores because the field url_key is ‘GLOBAL’ by default. For products, you can update the url_key attribute’s is_default field to ‘Store Views‘ from Attribute Management. But for the category’s url_key field, you don’t have that option in Magento. But the field does exist in the database with the same attribute code but having different backend_model so you need to go to the database directly and change the ‘is_global’ field 1 to 0.

Solution:
- First of all, access your Magento database from phpmyadmin (or any MySQL client).
- Then run the following query:

SELECT * FROM eav_attribute WHERE attribute_code LIKE '%url_key%';

- Then you will have following results:

479	9	url_key	NULL	catalog/category_attribute_backend_urlkey	varchar			text	URL key
481	10	url_key	NULL	catalog/product_attribute_backend_urlkey	varchar			text	URL key

- First attribute is for the category and second one is for the products. So note down the attribute_id of the first record (in my case 479).
- Then run the following query:

SELECT attribute_id, is_global FROM catalog_eav_attribute WHERE attribute_id=479;

To make sure if it has ’1′ by default or not because ’1′ is the value to make the attribute GLOBAL.

- If it is ’1′ then run the following query to update the value to ’0′:

UPDATE catalog_eav_attribute SET is_global='0' WHERE attribute_id=479;

- Now clean the cache and try to edit a category with different stores and see the field URL Key. This will now have ‘Store View’ scope.

Enjoy the SEO friendly URLs for categories too

In August 2010, I had developed script to connect to Magento server to pull some products by sending values to filter. All of a sudden, last Tuesday it stopped working. I had implemented the searching products from Magento via AJAX so the client was not able to see the error messages to. It seemed it very strange for me too.

Later when I try to connect to the Magento server from outside that AJAX script, then I found the following fatal error message:

Fatal error: Uncaught SoapFault exception: [WSDL] SOAP-ERROR: Parsing Schema: can't import schema from 'http://schemas.xmlsoap.org/soap/encoding/'

Magento normally contains the line following line in its WSDL:

And PHP SOAP was not being able to parse it and hence it gave the above error message.

I searched a lot but could not find the solid solution in Magento forum too. So I just opened the WSDL file located at /app/code/core/Mage/Api/etc/wsdl.xml and commented the line:

And tried as follows:

error_reporting(E_ALL & ~E_NOTICE);
ini_set('display_errors', 1);
$proxy      = new SoapClient('http://www.mysite.com/api/soap?wsdl');
$sessionId  = $proxy->login('****', '****');

Now it worked without any error message. Hope it helps you too!

Enjoy Magento SOAP!!!

I am working for the Magento API as a server and XML RPC PHP from Joomla as a client to bring up the information of product from Magento. Today I found where the problem persists and I have some workarounds for the fixes. But the fixes are in the Core files of Magento and Joomla itself.

Here is my detailed explanation and fixes for the problem.

Problem

When I was working with the XML RPC Magento API, everything working just fine like creating customers, listing product attribute sets and attributes, listing products, etc. But when I came to get the product information of a particular product in a page and used following code:

$client = new xmlrpc_client($path, $host, '');
//$client-->setDebug(true);
$login = new xmlrpcmsg('login', array(encode($apiUser), encode($apiKey)));
$doc = $client->send($login);
if(!is_object($doc)) {
    die('XML-RPC non-object document: ' . $doc);
}
$value = $doc->value();
if(!is_object($value)) {
    die('XML-RPC non-object value: ' . $value);
}
$session = $doc->value()->getval();
if(!$session -> 0) {
    die('Empty session ID: ' . $doc->faultString());
}
$options = new xmlrpcval(array(new xmlrpcval('837022070001', 'string')), 'array');
$msg = new xmlrpcmsg( 'call', array( encode($session), encode('product.info'), $options));
$doc = $client->send($msg);
if( $doc->faultCode() == 0 ) {
    $items = decode($doc->value());
    print_r($items);
}
else {
die($doc->faultString());
}

I keep receiving the following error message:

Invalid return payload: enable debugging to examine incoming payload found not-xmlrpc xml element NIL

When I tried the following to list the products, it just works fine:

$options = new xmlrpcval(array(
		new xmlrpcval(
			array(
				'status' =-> new xmlrpcval(array('eq' =-> new xmlrpcval('1', 'string') ), 'struct')
			), 'struct')
), 'array');
$msg = new xmlrpcmsg( 'call', array( encode($session), encode('product.list'), $options));
$doc = $client->send($msg);
if( $doc->faultCode() == 0 ) {
    $items = decode($doc->value());
   print_r($items);
}
else {
die($doc->faultString());
}

Actually this is kind of known bug in Magento specially in case of XML RPC API Server when it returns XML as a response. Magento returns an element empty <NIL> element if there is no value in the field. And the element <NIL> is not the valid element for XML RPC client parsers. It seems the problem is with all type of XML RPC client parsers for different languages like Java, C#, etc. becuase I can see lots of threads in magento discussion board about this but they are almost unsolved yet.

Solution

When I went through the manual/documentation of PHP XML RPC, there was some information under FAQ:

My client returns “XML-RPC Fault #2: Invalid return payload: enable debugging to examine incoming payload”: what should I do?

The response you are seeing is a default error response that the client object returns to the php application when the server did not respond to the call with a valid xmlrpc response.

The most likely cause is that you are not using the correct URL when creating the client object, or you do not have appropriate access rights to the web page you are requesting, or some other common http misconfiguration.

To find out what the server is really returning to your client, you have to enable the debug mode of the client, using

$client->setdebug(1);

Since I am working in Joomla 1.5 to get the product information from Magento, I am obviously using XML RPC client recommended by PHP itself ‘phpxmlrpc’ client library which is open source free library. So the problem was due to invalid XML element <NIL> sent by Magento as a response, when the parser tries to parse response XML, it terminates when it finds an invalid element <NIL> even though the XML is well formatted. So for now, I have modified the core phpxmlrpc client library file which is already packed with Joomla as a third party Library inside the library folder. I just went through all the classes and put following script to replace all the occurrences of <NIL> element before it validates the elements.

//File Modified :- libraries/phpxmlrpc/xmlrpc.php
//Line Number: 2539 (approx.)
//Joomla Version: 1.5.18
//XML RPC Version: xmlrpc.inc,v 1.158 2007/03/01 21:21:02 ggiunta Exp

// Date : July 29th, 2010
// Forcefully remove nil element - By Raju Gautam
// to overcome from the  element returned by Magento in response
// because the  element is not valid element for xml rpc
$data = str_replace(array('', '', '', '', '', ''), '', $data);

Though it cannot be assured that the script will work ahead without any problems because if there are other such invalid XML elements injected in any case by Magento API again then we have to replace those elements accordingly. I know the problem must be solved from Magento side but I fixed it from XML RPC client for now because I don’t have much knowledge with Magento yet.

Hope this will help anyone who is really running into the same problem and getting frustrated.

Good luck!

I was running into a problem while trying to use Magento web service to create a customer. Generally I have worked with Magento web services before and there were no such problems. But this time, I needed to create a customer in Magento with the information that is submitted from Joomla page. For this I installed Magento 1.4.1.0 in the server from command line without any problems. But as usual, when tried to access the WSDL

http://www.mysite.com/api/soap/?wsdl

but it just gave me a blank page neither it gave me any errors or warnings. I tried to Google and even tried Magento forums but no luck for 5/6 hours. I tried with different words but there was no luck.

It was just going weird for me and nothing to do except pulling my hair out and search and search. I was really stuck and was not being able to go ahead since I could not login to Magento through web service.

But later, I found a thread in Magento site from Google where someone faced and someone else suggested a solution for the same problem in same version of Magento. The problem is really a small but stupid one.:-)

In the following file app/code/core/Mage/Api/Model/Server/Adapter/Soap.php in around the line number 135 in my case inside the run() method, a closing curly brash was missed in the else part.

} else {
            $this->fault('0', 'Unable to load Soap extension on the server');
//<-- The curly brash was missed.

This file is the core file of Magento which handles the SOAP services. I don't know if the whole application was tested well then how this error was missed to fix. Otherwise have to say that they have missed testing in that file.

} else {
            $this->fault('0', 'Unable to load Soap extension on the server');
} //<-- The curly brash was missed.

When I fixed the error putting closing brash, it worked fine.

The use of good IDE for the PHP development is always good. A least anyone can get rid of small and syntax errors. Though it must be accidental case but if they have used some good IDE, the IDE would have thrown an error message inline.

Source: The solution was found in this thread http://www.magentocommerce.com/boards/viewthread/42062/