Showing posts with label ssis. Show all posts
Showing posts with label ssis. Show all posts

Thursday, March 29, 2012

about activeX script error in ssis package in SQL server 2005

when i run activex Script it's shows this error

[ActiveX Script Task] Error: Retrieving the file name for a component failed with error code 0x001B6438

Moving from .NET Framework Data Access and Storage...|||So is there a solution to this?
I've had a similar error when using the "scripting.filesystemobject" from within a script task in ssis.

about activeX script error in ssis package in SQL server 2005

when i run activex Script it's shows this error

[ActiveX Script Task] Error: Retrieving the file name for a component failed with error code 0x001B6438

Moving from .NET Framework Data Access and Storage...|||So is there a solution to this?
I've had a similar error when using the "scripting.filesystemobject" from within a script task in ssis.

Sunday, March 25, 2012

A/Synchronous execution of SSIS ETL packages

Hi

I'd like to know if there's a way to control the execution of ETL packages, such that:
Different packages, or at least packages that don't access the same table or database run asynchronously with respect to each other; e.g., two different packages run at the same time
and
If a package is called for execution more than once by different requests, force them to run synchronously, or one after the other.If this is possible, what resources would it require? Is this possible under, say, a dual or quad processor machine?

Thanks.

Well, there is nothing to stop you running 2 packages concurrently (i.e. at the same time). They run as seperate processes so cannot interfere with each other in the process space. Of course, if they are competing for the same external connections then you may run into problems.

Not so sure about ensuring that executions of a package are queued up tho as you require. You may have to implement a process on top of your packages to check to see if a package is already executing before attempting to execute it.

I don't think the spec of the machine is the issue here but it may help to know what resources a package utilises: http://blogs.conchango.com/jamiethomson/archive/2005/05/29/1486.aspx

-Jamie

|||

I'm not sure about the big picture, and how these package are run, but here are two ideas to consider:

1) using master package with execute package tasks and appropriate precedence constraints

2) creating Agent Job for each package - the Agent runs maximum one instance of a job at a time, thus if you always go via Agent, you'll have at most one instance of same package running.

|||

You would need to maintain some kind of shared state in order for packages in seperate processes to 'know' if a singleton object was being consumed by eachother and prevent contention. You can typically use a semaphore to stagger execution of any Windows dependant processes; in the SSIS context you could use a semaphore from within a script task. Have a look in the .Net framework for synchronization primitives support.

|||MSMQ task does this smashingly. :)

A Word about Meta-data, pass through columns and Derived columns

Here's another one of my bitchfest about stuff which annoy the *** out of me in SSIS (and no such problems in DTS):

Do you ever wonder how easy it was to set up text file to db transform in DTS - I had no problems at all. In SSIS - 1 spent half a day trying to figure out how to get proper column data types for text file - OF Course MS was brilliant enough to add "Suggest Types" feature to text file connection manager - BUT guess what - it sample ONLY 1000 rows - so I tried to change that number to 50000 and clicked ok - BUT ms changed it to 1000 without me noticing it - SO NO WONDER later on some of datatypes did not match. And boy what a fun it is to change the source columns after you have created a few transforms.

This s**hit just breaks... So a word about Derived Columns - pretty useful feature heh? ITs not f***ing useful if it DELETES SOME of the Code itself after there have been changes in dataflow. I cant say how pissed off im about that SSIS went ahead and deleted columns from flow & messed up derived columns just because the lineageIDs dont match.

Meta-data - it would be useful if you could change it and refresh it - im just sick and tired of it that it shows warnings and errors when there's nothing wrong - so after a change i need to doubleclick all my transforms so that those red & yellow boxes would disappear.

Oh and y I passionately dislike Derived columns - so you create new fields based on some data - you do some stuff - combine multiple columns to one, but you have no way saying remove the columns from the pipeline. Y you need it - well if you have 50K + rows with 30+ columns then its EXTRA useless memory overhead for your package.

Hopefully one day I will understand how SSIS works (not an ez task I say) - I might be able to spend more time on development and less time on my bitchfest - UNTIL then --> Another Day - Another Hassle with SSISOh - i also forgot - trying to get to columns to appear in the right order is no picnic either. If you are dealing with 80+ you would like them to always appear on same order (alphabetic or otherwise meaningful) but SSIS seems to shuffle the order also every now and then|||

TheViewMaster wrote:

Here's another one of my bitchfest about stuff which annoy the *** out of me in SSIS (and no such problems in DTS):
Do you ever wonder how easy it was to set up text file to db transform in DTS - I had no problems at all. In SSIS - 1 spent half a day trying to figure out how to get proper column data types for text file - OF Course MS was brilliant enough to add "Suggest Types" feature to text file connection manager - BUT guess what - it sample ONLY 1000 rows - so I tried to change that number to 50000 and clicked ok - BUT ms changed it to 1000 without me noticing it - SO NO WONDER later on some of datatypes did not match. And boy what a fun it is to change the source columns after you have created a few transforms.

If you're going to rely on a computer to "guess" what the column data-types should be then you deserve all that's coming to you. "Suggest Types" is just that. A suggestion. It is your responsibility to make sure the column data-types are correct.

TheViewMaster wrote:


This s**hit just breaks... So a word about Derived Columns - pretty useful feature heh? ITs not f***ing useful if it DELETES SOME of the Code itself after there have been changes in dataflow. I cant say how pissed off im about that SSIS went ahead and deleted columns from flow & messed up derived columns just because the lineageIDs dont match.

It doesn't delete columns itself. Only if you do something to cause that. Lineage IDs are vital to how the pipeline engine works - this behaviour is absolutely correct and there are justifiable reasons for it.

I have never seen it delete code without there being some perfectly good, user-produced, reason.

TheViewMaster wrote:


Meta-data - it would be useful if you could change it and refresh it - im just sick and tired of it that it shows warnings and errors when there's nothing wrong - so after a change i need to doubleclick all my transforms so that those red & yellow boxes would disappear.

You can. If downstream components are still affected then it will flag this to you. What you will probably find is that it won't automatically fix components that are downstream of asynchronous components. That is absolutely correct. What SSIS will not do is make changes without telling you about it - it expects you to make the decisions, it will not make decisions for you. In this regard it is a huge improvement from DTS which would often make guesses about things which could be wrong.

TheViewMaster wrote:


Oh and y I passionately dislike Derived columns - so you create new fields based on some data - you do some stuff - combine multiple columns to one, but you have no way saying remove the columns from the pipeline. Y you need it - well if you have 50K + rows with 30+ columns then its EXTRA useless memory overhead for your package.

Wrong again. Removing columns from memory (and all the subsequent memory management that must occur) at execution time after they are no longer required was considered to be too detrimental an overhead in terms of pipeline performance. The pipeline is there to produce screamingly fast data movement and anything that impacts that (such as removing unused columns) will not be done.

TheViewMaster wrote:


Hopefully one day I will understand how SSIS works (not an ez task I say)

Absolutely correct. Its not easy - its a huge product and there's loads to understand. There is a learning curve with SSIS as there is any other product. If you're not prepared to go through that curve then that's fine - don't use the product. But don't moan about it endlessly.

TheViewMaster wrote:

- I might be able to spend more time on development and less time on my bitchfest - UNTIL then --> Another Day - Another Hassle with SSIS

I don't really know why I'm bothering replying. You obviously have decided that SSIS is useless without even trying to understand the justification for any of these behaviours. I'm biting my tongue till it hurts, believe me.

I hope as you come to learn more about SSIS (if you choose to continue down this route) you will understand some of the justifications for the behaviour that is inherent in the UI. There are definately improvements to be made, I won't disagree with you there, and if you have legitamate requests for enhancements (such as your comment about reordering of columns) then there are proper channels for submitting those requests. Whinging on this forum is not one of them.

-Jamie

|||1000 rows is the scan limit of the "suggest types" feature.

Aren't you prompted to fix invalid metadata when changes upstream are made?

I hear you on being able to drop columns from the dataflow whenever you'd like. That would be a nice feature.

But I have to ask... With your growing list of issues with SSIS, I wonder:
- Have you filled out bug/feature requests?
- Is SSIS the right tool for you? Perhaps something like Informatica would be better suited for you.

Phil|||

TheViewMaster wrote:

Oh - i also forgot - trying to get to columns to appear in the right order is no picnic either. If you are dealing with 80+ you would like them to always appear on same order (alphabetic or otherwise meaningful) but SSIS seems to shuffle the order also every now and then

SSIS indeed does not gaurantee column ordering. SSIS will try to maintain order from the source but there are cases where it won't. For example if you go through an asynchronous component then the ordering is however that component puts it on the output since it is effectively a "new source".

Thanks
Mark

|||

Jamie Thomson wrote:


I don't really know why

I'm bothering replying. You obviously have decided that SSIS is useless

without even trying to understand the justification for any of these

behaviours. I'm biting my tongue till it hurts, believe me.

Please dont stop - I am trying to understand how things work - it's just it can be frustrating spending too much time on stuff - this forum has deffinately being very useful thanks to you and all other responders

Phil Brammer wrote:

1000 rows is the scan limit of the "suggest types" feature.

Aren't you prompted to fix invalid metadata when changes upstream are made?

I hear you on being able to drop columns from the dataflow whenever you'd like. That would be a nice feature.

Yes - i figured it out - thank god I had backup copy of the version of SSIS package somewhere. The issue is that i changed some transformation (instead of merge join - a lookup) and that caused LineageIDs to not match. and in the transform to rawfile - there were columns like CITY_NAME renamed to CityName - and of course SSIS wasnt able to match them by name. So even with finding that piece of tranformation from backup i still had to do a lot of manual work to get all columns back.

- Have you filled out bug/feature requests?


- I tried once & only once - its a hassle to fill it out

- Is SSIS the right tool for you? Perhaps something like Informatica would be better suited for you.


It is "free" - so it must be the right one.
The learning curve is tough on that one - reminds me of moving from vb6/asp to .Net = a lot of hangst and head-banging.
But I start to see the light & still the dev model of SSIS outweighs the hurdles with some parts

Tuesday, March 20, 2012

A truncation occurred during evaluation of the expression

First of all, I get the following error message for one of my packages which uses user variables:

SSIS package "UsageAnalysis.dtsx" starting.
Information: 0x4004300A at Perform xmlState Shredding, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Error: 0xC001700E at UsageAnalysis:
A truncation occurred during evaluation of the expression.
Error: 0xC0019004 at UsageAnalysis: The expression for variable "GetAnalysisData" failed evaluation. There was an error in the expression.
Error: 0xC02020E9 at Update Analysis Table, UsageAnalysis Source [1]: Accessing variable "User::GetAnalysisData" failed with error code 0xC001700E.
Error: 0xC0024107 at Update Analysis Table: There were errors during task validation.
Warning: 0x80019002 at Usage Analysis Process: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "UsageAnalysis.dtsx" finished: Failure.

Now my package has the following variables:

GetMaxUsageID: scope package level, type string, statement SELECT MAX(UsageID) AS MaxUsageID FROM XX.XXX

MaxUsageID: scope package level, type int32, default value 0, value get assigned from the following statement executed from sql task that runs GetMaxUsageID variable as above

GetAnalysisData: scope package level, type string, Evaluate as Expression

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 8, 1252) @.[User::MaxUsageID]

The package has worked fine until MaxUsageID value reached to 10,00,000 and since then I have been getting above mentioned error message. The problematic step is related to Data Flow task where I use GetAnalysisData. I have tried replacing user variable with literal as follows

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

the error message stays the same. Please note that package has worked fine before and it still works ok if I don't use user variables. Obviously, some of you would see eliminating user variables as workaround but I would appreciate if cause of that error message could be investigated.

Thanks,

Asaf

Are you sure MAX(UsageID) is 1,000,000 and not 100,000,000?

Try making your cast a longer length in the GetAnalysisData variable. Perhaps: (DT_STR,20,1252)@.[User::MaxUsageID]

|||

Hi Phil,

Thanks for your reply but I am afraid I already have tried the solution you suggested. Max(UsageID) is calculated against sql table and then stored to MaxUsageID user variable.

select max(usageID) from dbo.XXX

--1027005

So you could see value by no means is tens of millions as you anticipated. I also have tried changing DT_STR string length parameter to larger values as proposed

SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 20, 1252) @.[User::MaxUsageID]

Even when I have no casting involved such as

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

I still get the same truncation error message. I would appreciate if you could suggest something else which obviously I am missing here.

Thanks,

Asaf

|||

Phil,

I have worked it out.

It looks like one of the SSIS bugs. Even though I updated the expression by changing (DT_STR, 6, 1252) to (DT_STR, 12, 1252) so that I could avoid any obvious truncation issues for values above 1 million and after saving the package I rebuilt it but project would fail whenever I executed it.

When I opened the project again, I ended up having the old value for (DT_STR, 6, 1252) in xml code view. Somehow SSIS wouldn't save the updated expression. Hence I kept getting that error message even though I had reasons to believe I had done everything right.

To resolve that, I just decided to define a new variable with required DT_STR string length and my package works without any complain.

Thanks,

Asaf

|||

Sounds like you may have been trying to update the "Value" parameter, not the "Expression" parameter. In that case, it would revert back to whatever you had in the expression.

With that being said, if it is a bug and you can reproduce it in the future, please submit it at http://connect.microsoft.com/sqlserver/feedback and report back here with the link to your submission. But please only submit it if you can reproduce it.

Thanks,

Phil

sql

A truncation occurred during evaluation of the expression

First of all, I get the following error message for one of my packages which uses user variables:

SSIS package "UsageAnalysis.dtsx" starting.
Information: 0x4004300A at Perform xmlState Shredding, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Error: 0xC001700E at UsageAnalysis:
A truncation occurred during evaluation of the expression.
Error: 0xC0019004 at UsageAnalysis: The expression for variable "GetAnalysisData" failed evaluation. There was an error in the expression.
Error: 0xC02020E9 at Update Analysis Table, UsageAnalysis Source [1]: Accessing variable "User::GetAnalysisData" failed with error code 0xC001700E.
Error: 0xC0024107 at Update Analysis Table: There were errors during task validation.
Warning: 0x80019002 at Usage Analysis Process: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "UsageAnalysis.dtsx" finished: Failure.

Now my package has the following variables:

GetMaxUsageID: scope package level, type string, statement SELECT MAX(UsageID) AS MaxUsageID FROM XX.XXX

MaxUsageID: scope package level, type int32, default value 0, value get assigned from the following statement executed from sql task that runs GetMaxUsageID variable as above

GetAnalysisData: scope package level, type string, Evaluate as Expression

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 8, 1252) @.[User::MaxUsageID]

The package has worked fine until MaxUsageID value reached to 10,00,000 and since then I have been getting above mentioned error message. The problematic step is related to Data Flow task where I use GetAnalysisData. I have tried replacing user variable with literal as follows

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

the error message stays the same. Please note that package has worked fine before and it still works ok if I don't use user variables. Obviously, some of you would see eliminating user variables as workaround but I would appreciate if cause of that error message could be investigated.

Thanks,

Asaf

Are you sure MAX(UsageID) is 1,000,000 and not 100,000,000?

Try making your cast a longer length in the GetAnalysisData variable. Perhaps: (DT_STR,20,1252)@.[User::MaxUsageID]

|||

Hi Phil,

Thanks for your reply but I am afraid I already have tried the solution you suggested. Max(UsageID) is calculated against sql table and then stored to MaxUsageID user variable.

select max(usageID) from dbo.XXX

--1027005

So you could see value by no means is tens of millions as you anticipated. I also have tried changing DT_STR string length parameter to larger values as proposed

SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 20, 1252) @.[User::MaxUsageID]

Even when I have no casting involved such as

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

I still get the same truncation error message. I would appreciate if you could suggest something else which obviously I am missing here.

Thanks,

Asaf

|||

Phil,

I have worked it out.

It looks like one of the SSIS bugs. Even though I updated the expression by changing (DT_STR, 6, 1252) to (DT_STR, 12, 1252) so that I could avoid any obvious truncation issues for values above 1 million and after saving the package I rebuilt it but project would fail whenever I executed it.

When I opened the project again, I ended up having the old value for (DT_STR, 6, 1252) in xml code view. Somehow SSIS wouldn't save the updated expression. Hence I kept getting that error message even though I had reasons to believe I had done everything right.

To resolve that, I just decided to define a new variable with required DT_STR string length and my package works without any complain.

Thanks,

Asaf

|||

Sounds like you may have been trying to update the "Value" parameter, not the "Expression" parameter. In that case, it would revert back to whatever you had in the expression.

With that being said, if it is a bug and you can reproduce it in the future, please submit it at http://connect.microsoft.com/sqlserver/feedback and report back here with the link to your submission. But please only submit it if you can reproduce it.

Thanks,

Phil

Sunday, March 11, 2012

A SSIS package

Hi,

I am used Visual Studio SSIS wizard to transfer some data from one database to another with the same table structure. This is the first time I use SSIS. I see two objects created. OLE DB Source extracts some data based on the create date and OLE DB Destination object is a corresponding table.

So query in OLE DB Source:

Select ID,[Desc],[CreateDate] from TableSrc where ],[CreateDate] between ‘1/1/2006’ and ‘1/31/2006’

OLE DB Destination has TableDest as the destination. TableDest has the same structure as TableSrc.

My problem is that when I run the package twice the data will be imported twice. I need to use this package for both new records and updated records in TableSrc . Is there any way I can check if the ID is available in the TableDest, I perform update otherwise perform insert into TableDest.

Thanks,

That is a verry common scenario when loading data. The most popular solution in this forum is tu used a Lookup transform in the data flow against the destination table; no matches in LU transform are treated as errors; so you can configure the error output of the LU transform to 'redirect error'; then the error output is your 'new rows' out put and the no-error output is the 'existing rows' output.:

Somewhere in this thread there is a link to Jamie's blog where that technique is explanied (an other options discussed).

http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=779836&SiteID=17

Be sure to understand how Lookup transform uses memory resources as that could play against you if the volume of data is to big or if memory not enough

|||I second Rafael's comments.

A SSIS package

Hi,

I am used Visual Studio SSIS wizard to transfer some data from one database to another with the same table structure. This is the first time I use SSIS. I see two objects created. OLE DB Source extracts some data based on the create date and OLE DB Destination object is a corresponding table.

So query in OLE DB Source:

Select ID,[Desc],[CreateDate] from TableSrc where ],[CreateDate] between '1/1/2006' and '1/31/2006'

OLE DB Destination has TableDest as the destination. TableDest has the same structure as TableSrc.

My problem is that when I run the package twice the data will be imported twice. I need to use this package for both new records and updated records in TableSrc . Is there any way I can check if the ID is available in the TableDest, I perform update otherwise perform insert into TableDest.

Thanks,

I can't find a simple way to do this in SSIS. I'd better do this in T-SQL, usingLinked Servers. Suppose you've set a linked server (namedSourceServer) for the source server on destination server, you can use such query to accomplish INSERT/UPDATE:

UPDATE TableDest
SET [Desc]=src.[Desc], [CreateDate]=src.[CreateDate]
FROM TableDest dest JOIN [SourceServer].SourceDB..TableSrc src
ON dest.ID=src.ID
WHERE?src.[CreateDate] between '1/1/2006' and '1/31/2006'

INSERT INTO TableDest
SELECT * FROM [SourceServer].SourceDB.TableSrc src
WHERE?src.[CreateDate] between '1/1/2006' and '1/31/2006'
AND src.ID NOT IN (SELECT ID FROM TableDest)

a solution

Hi,

I know this issue exisits in DTS but needs to check still is in SSIS, Also you guys may have a better solution for it.

Issue: When I try to import a column from excel which has data like A,B,C,D,E,4,5 in the destination table has the data type as varchar it imports only A,B,C,D,E and 4 & 5 as nulls. How to fix this.

Set the Excel connection Extended Property, IMEX=1.

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1294377&SiteID=1|||Thanks, it worked. Appreciated.|||

Imex=1 worked for all the cells except few, this column has data like below

07AWA38V36 0717062042 715018020 07BCB01R17

even in this column some has been imported correctly and few are imported like 7.15E+...

Note: My desination column data type in Varchar.

|||Make sure that Excel doesn't have any formatting for the problem cells or the problem column.|||In the above example for the second row the data was formatted as text becasue it has the leading Zero where as the third is not formatted becuase it doesnt have any leading zero. the issue was in the third row coverting as 7.15E+.. while running the SSIS|||

yes, i′ve got just the same problem.

incredibly it didn′t happen the first time i runned the dts, but now..

|||

..and i just got it..

I selected the whole worksheek, converted the cells into numeric type (format/cells/numeric) and saved. Then i converted the cells into General type again, saved and runned the dts. And it works now.

Better not to use Text types when it happens. When you′ve got a Text type it doesnt work properly, and if you change to General => it doesnt work either. But if you had Numeric types and change to General then it works.

|||

When I change this to Numeric then I will loose the leading zero in the 2 & 3 row.

|||

? no, you wont..

It′s just an excel fail, i doesnt catch properly that there′s a number, not a text, when you had a text previously

a solution

Hi,

I know this issue exisits in DTS but needs to check still is in SSIS, Also you guys may have a better solution for it.

Issue: When I try to import a column from excel which has data like A,B,C,D,E,4,5 in the destination table has the data type as varchar it imports only A,B,C,D,E and 4 & 5 as nulls. How to fix this.

Set the Excel connection Extended Property, IMEX=1.

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1294377&SiteID=1|||Thanks, it worked. Appreciated.|||

Imex=1 worked for all the cells except few, this column has data like below

07AWA38V36 0717062042 715018020 07BCB01R17

even in this column some has been imported correctly and few are imported like 7.15E+...

Note: My desination column data type in Varchar.

|||Make sure that Excel doesn't have any formatting for the problem cells or the problem column.|||In the above example for the second row the data was formatted as text becasue it has the leading Zero where as the third is not formatted becuase it doesnt have any leading zero. the issue was in the third row coverting as 7.15E+.. while running the SSIS|||

yes, i′ve got just the same problem.

incredibly it didn′t happen the first time i runned the dts, but now..

|||

..and i just got it..

I selected the whole worksheek, converted the cells into numeric type (format/cells/numeric) and saved. Then i converted the cells into General type again, saved and runned the dts. And it works now.

Better not to use Text types when it happens. When you′ve got a Text type it doesnt work properly, and if you change to General => it doesnt work either. But if you had Numeric types and change to General then it works.

|||

When I change this to Numeric then I will loose the leading zero in the 2 & 3 row.

|||

? no, you wont..

It′s just an excel fail, i doesnt catch properly that there′s a number, not a text, when you had a text previously

a solution

Hi,

I know this issue exisits in DTS but needs to check still is in SSIS, Also you guys may have a better solution for it.

Issue: When I try to import a column from excel which has data like A,B,C,D,E,4,5 in the destination table has the data type as varchar it imports only A,B,C,D,E and 4 & 5 as nulls. How to fix this.

Set the Excel connection Extended Property, IMEX=1.

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1294377&SiteID=1|||Thanks, it worked. Appreciated.|||

Imex=1 worked for all the cells except few, this column has data like below

07AWA38V36 0717062042 715018020 07BCB01R17

even in this column some has been imported correctly and few are imported like 7.15E+...

Note: My desination column data type in Varchar.

|||Make sure that Excel doesn't have any formatting for the problem cells or the problem column.|||In the above example for the second row the data was formatted as text becasue it has the leading Zero where as the third is not formatted becuase it doesnt have any leading zero. the issue was in the third row coverting as 7.15E+.. while running the SSIS|||

yes, i′ve got just the same problem.

incredibly it didn′t happen the first time i runned the dts, but now..

|||

..and i just got it..

I selected the whole worksheek, converted the cells into numeric type (format/cells/numeric) and saved. Then i converted the cells into General type again, saved and runned the dts. And it works now.

Better not to use Text types when it happens. When you′ve got a Text type it doesnt work properly, and if you change to General => it doesnt work either. But if you had Numeric types and change to General then it works.

|||

When I change this to Numeric then I will loose the leading zero in the 2 & 3 row.

|||

? no, you wont..

It′s just an excel fail, i doesnt catch properly that there′s a number, not a text, when you had a text previously

Thursday, February 9, 2012

A good SSIS book?

I am new to SQL2005 and have been given the task of writing some SSIS packages to import some CSV files.

I need to cleans the data as it is imported from my CSV files before it reaches my SQL DB.

I am currently Googling the internet to discover how to do this.

Can anyone recommend a good SSIS book?

I am a C# developer, so a book that has lots of SSIS C# examples would be good.

Any help appreciated.

Regards,

Paul.

SSIS(SQL Server Integration Services) is a full ETL(extraction transformation and loading) tool so moving CVS is very easy task I don't think you need a book to move CVS files to SQL Server because the only issue with CVS files is SQL Server sees null values due to the nature of the file. The solution is to move the file to a temp table before destination because if you move it directly to a table with primary key your package will be rejected. I have found you a two part article by Microsoft and the site run by first DTS and now SSIS experts, this is the calculus end of the relational model, the data is cleaned of algebra and moved to read only tables for Dimension modeling and Prediction modeling. I will not recommend a book because most things on the calculus end of SQL Server is new so you need to browse some books at your local book store and choose the one that meets your needs I found three books on the service. Post again if you still have questions. Hope this helps.

http://msdn2.microsoft.com/en-us/library/aa964134.aspx

http://www.sqlis.com/

|||

I have ordered a SSIS book from Amazon - Looks like I'm a steep learning curve.

Thanks for the help.

Regards,

Paul.

|||

wadep:

I have ordered a SSIS book from Amazon - Looks like I'm a steep learning curve.

Thanks for the help.

Regards,

Paul.

I am glad I could help, just remember to start with the wizard and use code as needed.